acceptodds
Under review as a conference paper at ICLR 2027

Language Models can Learn High-Capacity Secure Steganography

Abstract

Steganography hides messages inside innocuous output so that an observer cannot tell a secret is being sent. This is a safety concern for language models deployed under monitoring. The worst case for a defender is a model whose output is statistically indistinguishable from its normal generation, yet hides as much secret information per token as information theory allows. In this work, we show that language models can be trained to approximate such schemes, and we study the signatures they leave for monitors. We introduce MEC-LLM, an LLM model organism trained using objectives motivated by minimum-entropy coupling (MEC). The trained model encodes uniformly random 128-bit secrets at a raw rate of 0.57 bits per token. A classifier using only text features distinguishes stego from cover with AUC 0.62, while one using the model's full output logits reaches AUC 1.00. We also show that defenders can disrupt the channel by paraphrasing the stegotext with Claude Haiku 4.5: across 13 paraphrase prompts ranging from minimal edits to structural rewrites, bit-error rate rises from 12.8% to 48.7%. Finally, we introduce a complete model organism that pairs MEC-LLM with a trained payload generator to exfiltrate secrets planted in the system prompt, giving AI safety researchers a controlled target for studying internalized covert channels. We release this model organism for future work on monitoring and interpretability of covert communication in language models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
steganography, large language models, AI safety, minimum-entropy coupling, covert communication, model organisms

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_6sVYOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the use of LLM to learn secure steganographic encoding and decoding algorithms that have already been proposed. It transforms the traditionally external algorithm-based information hiding process into a capability that can be realized through neural network training, and uses experiments to demonstrate that LLM can learn high-capacity steganographic communication mechanisms, that internal model states can be used to detect the existence of steganography, and that the robustness of the generated cipher texts generated in this way has also been evaluated. The authors indeed trained a Transformer module capable of performing steganographic encoding and decoding, but in my view, the paper does not prove, as the title claims, that an LLM originally lacking steganographic capabilities can learn steganography through conventional pre-training, SFT, or RL post-training.

Critical strengths

This paper transforms the optimization objective of traditional Minimum-Entropy Coupling into a differentiable loss function, jointly optimizing steganographic capacity, distribution matching, and secret decoding capability, enabling neural networks to directly learn high-capacity, low-detectable text steganography mechanisms without relying on pre-generated Stego training labels or explicit MEC encoding algorithms.

Critical weaknesses
  1. This paper is more about proposing a neural network encoder-decoder similar to the classic HiDDeN method, but it deals with the hidden states and token probability distributions of a language model rather than text or images. The paper does not prove that the LLM learned steganography. It can only indicate that the newly added module enables the LLM to achieve steganography similar to MEC.
  2. The title directly uses "Secure Steganography"; thus, it is easy to form further expectations: the mechanism learned by the model has the security properties defined in the paper. The author adopts the theoretical objective of MEC to train a neural encoder, but the probability distribution learned does not meet the strict safety constraints required by MEC.
Questions for authors

Why do the authors adopt an append-only Transformer architecture rather than directly fine-tuning the pretrained LLM using the same MEC objective? Could you provide comparisons with LoRA or full-parameter fine-tuning under the same dataset, mechanism, and loss functions?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 6sVY (1/2)

We thank the reviewer for their feedback. We hope the results below address the reviewer's concerns and respectfully ask whether they would consider raising their score.

Weakness 1: Did the LLM learn steganography?

The paper does not prove that the LLM learned steganography. It can only indicate that the newly added module enables the LLM to achieve steganography similar to MEC.

We want to answer this question from three different angles:

  1. Can a transformer learn high-capacity secure steganography? MEC is np-hard and undetectable from output text which raises the questions whether transformers have the capability to express and compute this at all and if there are any limits. To answer this question, we trained a transformer with an encoder-decoder method and loss. We showed that transformers can learn this and identified limitations like detectability in logit space and the recursive nature of updating a belief token-by-token without feeding the belief back in into the model.

  2. Did the LLM itself learn steganography or is it only enabled through the added module?

The appended layers are ordinary transformer layers and simply extend the existing LLM. There is no architectural difference between the added and existing layers except in how they were trained. Taken together, the new model is a single LLM which can perform steganography.

However, we added new experiments that show that the LLM itself can directly learn MEC steganography, without adding a new module. We show in our answer below to Question 1 that it is possible to directly finetune the LLM using the same method and loss to make the base model itself do steganography.

  1. Can an LLM learn high-capacity secure steganography during ordinary pretraining, sft, or RL?

encoder-decoder similar to the classic HiDDeN method

First, we think certain RL posttraining can create similar pressures. During RL, an LLM might be monitored (creating security pressure) and get rewarded if it solves a task by passing messages the monitor doesn’t flag (creating decode accuracy pressure).

Second, we added an experiment showing that LLMs can learn secure steganography with SFT. We generated an SFT dataset by sampling 2 million completions with MEC-LLM and with the external greedy MEC sampler. We then finetuned Qwen2.5-1.5B with a learning rate of 1e-5 and cosine decay and 160 sequences per step. Because this sft’ed model doesn’t have a belief head and we thus can’t read out what it decodes, we strip the key of some training data and append it to the model’s response. This teaches the model to decode the secret from text and output it as tokens. We found that Qwen2.5-1.5B can learn to encode and decode through ordinary SFT when trained on a dataset generated with MEC-LLM. Interestingly, it couldn’t learn from the external greedy MEC sampler, likely because its encoding has no consistent pattern: which token encodes which secret bits depends on tiny probability differences that change from context to context.

Taken together, these experiments show that LLMs can learn high-capacity secure steganography without added modules or custom loss functions.

SFT data fromTeacher (%)Student, decoded by the teacher's receiver (%)Student, decoded by itself (%)
MEC-LLM86.456.9–
MEC-LLM, plus a decoding task‡86.063.864.5
External greedy MEC82.0* / 43.8†6.7–

Weakness 2: Strict security constraints for MEC-LLM:

The title directly uses "Secure Steganography"; thus, it is easy to form further expectations: the mechanism learned by the model has the security properties defined in the paper. The author adopts the theoretical objective of MEC to train a neural encoder, but the probability distribution learned does not meet the strict safety constraints required by MEC.

We appreciate the reviewers request for mathematical precision. Below, we disambiguate different levels of security, show that MEC-LLM is close to but not perfectly at the mathematical optimum, and that the resulting models are difficult to detect. We also have improved the paper with the following points.

Following Cachin et al. 1998, we distinguish between three levels of security:

  • “Perfectly secure”: Exact distributional equality between cover and stegotext, KL(cover||stego) = 0
  • “ε-secure”: Approximate distributional equality, KL(cover||stego) < ε.
  • “Insecure”: Stegotext distribution is different from covertext.

Our training directly optimizes for perfect security but the approximate nature of neural network optimization prevents us from reaching exact “perfect security”. Instead, MEC-LLM is ε-secure with a very small ε of around ε=0.15 nats.

AuthorsOpenReview ↗

Response to Reviewer 6sVY (2/2)

We also measure security empirically using various blackbox detection methods in section 4.4 and show that our models are hard to detect even with affordances like having access to a reference model and a labelled dataset. In practice we won’t have those affordances and that will make detection even more difficult.

Finally, we compared security empirically with other publicly available stegomodels. We compute ε-security for TrojanStego by Meier et al. 2025 and for MEC-LLM. We observe that KL divergence is 7-14 times higher for comparable payload rates.

MethodBase modelPayloadRate (bits/tok)ε (nats) ↓
MEC-LLM (ours)Qwen2.5-3B128 bit1.820.148
MEC-LLM (ours)Qwen2.5-3B-Instruct128 bit0.740.152
TrojanStegoMinistral-8B-Instruct32 bit1.001.508
TrojanStegoLlama-3-8B-Instruct32 bit1.002.125

Taken together, we think that MEC-LLM is secure and much more so than other models, and we have edited the paper to disambiguate the meaning of security better.

Question 1: Full-weight finetune

Why do the authors adopt an append-only Transformer architecture rather than directly fine-tuning the pretrained LLM using the same MEC objective?

In order to simplify training, but directly finetuning the pretrained LLM is possible too: Training a few appended layers is cheaper and faster than finetuning the whole model within our compute budget, and reference activations can be cached and reused across training runs. Also, MEC-LLM needs to learn to inspect and shape its own output distribution. Therefore, a natural choice is to compute the actual output distribution (the final residual stream activations) first using the frozen reference model, then to pass this to MEC-LLM with the secret and train it to conditionally shape it. Finally, we hypothesized that finetuning the pretrained LLM directly might be harder because the circuits implementing MEC need to share parameters and activations with the reference model’s computation. They also need to be computed before the reference model finishes computing the unconditional output distribution.

However, appending transformer layers is not a fundamental requirement. To show that directly fine-tuning the pretrained LLM is possible too, we trained two models using the same MEC objective, one with appended layers and frozen backbone (control, as in the paper) and a full-weight finetune of the underlying model. The results are in the table below. Overall, both approaches work and have comparable accuracy and security.

ModelStepsBlock accuracySecurity KL, eval (nats/token)Non-encoding KL (nats/token)
Appended layers100,00060.0%0.3580.006
Full fine-tune100,00057.2%0.2230.100

Question 2: LoRA

Could you provide comparisons with LoRA or full-parameter fine-tuning under the same dataset, mechanism, and loss functions?

Yes, we trained Qwen2.5-1.5B-Instruct with LoRA and full-parameter finetuning using the same dataset, trainer, and loss functions for 84 B300-hours (to match compute, LoRA was slightly slower) and 192 sequences per batch. For LoRA, we used rank 64 on every projection and a learning rate of 2e-4 following “LoRA Without Regret”. Both work and their performance is comparable. LoRA converges towards slightly worse decode accuracy but slightly better security, likely because we force the model to change fewer parameters:

MethodStepsBlock accuracySecurity KL (nats/token)
Full fine-tuning23,00075.7%0.173
LoRA (rank 64)17,00069.2%0.137

I sincerely thank the authors for their detailed and thoughtful responses. The clarifications and additional experiments have addressed my main concerns to a large extent.

Upon further reflection, I recognize that my initial assessment may have placed too much emphasis on the distinction between the frozen backbone and the newly added components. Architectural modifications are an integral part of language model development, and I agree that the original work demonstrates that a language model with specialized architectural design and optimization can learn to perform steganography. The additional fine-tuning and LoRA experiments further strengthen this conclusion by showing that language models can acquire steganographic encoding and decoding capabilities without the proposed architectural extensions. These results are particularly valuable, as they directly address the broader learning capability that I had hoped to see investigated in the original submission.

Overall, I appreciate the authors' efforts in addressing these concerns, and the new evidence has positively influenced my assessment of the paper. In particular, I believe the new fine-tuning results significantly broaden the implications of the work by providing evidence that language models can acquire steganographic capabilities without dedicated architectural modifications. I encourage the authors to incorporate these additional experiments into the revised manuscript and further discuss their implications for how ''language models can learn steganography.''

We thank the reviewer for engaging with our rebuttal on such a thoughtful and detailed level, and for increasing their score.

We have now uploaded the revised paper and marked all changed in red for readability.

Importantly, we addressed the reviewers two critical weaknesses in the paper's main body: In the introduction, background, and throughout, we differentiate between perfect and approximate security and highlight our reasoning and experiments around why approximate security with small epsilon is achieved and sufficient.

We also added section 6, where we discuss in greater detail how LLMs can learn MEC steganography in natural settings. Foremost, we show that LLMs can learn MEC steganography using ordinary SFT on the full model without appending layers. This suggests that adding chats encoded with MEC-LLM is sufficient to make an ordinary LLM learn to perform steganography during SFT.

Finally, we added the remaining rebuttal experiments in Appendix M and throughout the main section paper.

We would be grateful if the reviewer would consider raising their score to recommend acceptance, or let us know if any remaining concerns prevent them from doing so. We thank the reviewer again for their time and constructive feedback.

Reviewer_GkVUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the AI-safety risk that a language model could hide information in its ordinary-looking output so that a collaborator can recover it, and asks whether a model can internalize a scheme that is both statistically undetectable and close to the information-theoretic capacity limit. The authors train additional layers on top of a frozen open-weight model against objectives motivated by minimum-entropy coupling, and pair the result with a component that selects what to encode from the prompt, forming a model organism intended as a target for monitoring and interpretability research.

Critical strengths
  1. The threat model is well motivated, and studying the worst case is the right framing for defenders.

  2. The defense analysis is the most useful part, giving concrete guidance on logit-level monitoring and paraphrasing.

  3. The reporting is candid, openly conceding where the trained model falls short of an external baseline.

Critical weaknesses
  1. The central security claim is asserted rather than measured. The paper derives a sequence-level security divergence but never reports it for the evaluated model, and the only divergence it gives is a diagnostic the authors say does not measure security. Meanwhile, text-only classifiers separate stego from cover above chance, and strongly so for the chat model organism, so the headline claim of secure steganography is not supported.

  2. The channel does not work end to end. Payload recovery is unreliable, almost no complete secret is recovered, capacity is about half the target, and no error-correcting code is tested. The authors' own appendix shows that an external implementation is better on every comparable axis, so the trained model is not the worst-case adversary the framing promises.

  3. The theory does not describe the evaluated model. The released checkpoint was trained with a four-term objective that, by the paper's own account, does not estimate the sequence-level security divergence developed in Section 3.2 and Appendix L, and the per-block posterior reset is conceded to hold only under assumptions the architecture does not satisfy.

Questions for authors
  1. What is the sequence-level security divergence of the evaluated checkpoint?
  2. Was the released checkpoint trained with the objective of Section 3.2, and if not, do its security results apply?
  3. How much payload survives at the reported error rate once an error-correcting code is applied?
  4. What context does the receiver need to decode, and does it include the secret-bearing prompt?
4 comments
AuthorsOpenReview ↗

Response to Reviewer GkVU (1/2)

We thank the reviewer for identifying these important points. We ran additional experiments and clarified the distinction between theoretical MEC, its neural approximation, and the empirical performance of the trained models. We are grateful the reviewer found that “the threat model is well motivated”, “defense analysis is the most useful part”, and “reporting is candid”, and ask whether they would consider raising their score.

Weakness 1: Security.

The paper derives a sequence-level security divergence but never reports it for the evaluated model

Here is the (per-token) KL divergence between cover- and stegotext:

MethodBase modelPayloadKL (nats/token) ↓
MEC-LLM (ours)Qwen2.5-3B128 bit0.148
MEC-LLM (ours)Qwen2.5-3B-Instruct128 bit0.152

We also addressed the strong text-only detectability of the conversational model. We found that this was caused by training on blocks with too few tokens. Even a perfect MEC encoder like external greedy MEC couldn’t encode all bits inside the short blocks and MEC-LLM was trading security for decode accuracy. We retrained MEC-LLM-1.5B with 8 and 16 tokens per block and found that more tokens significantly improved security at similar or slightly better accuracy:

Detector AUC ↓8 tokens/block16 tokens/block
Text + PPL0.890.75
DistilBERT0.880.77
Text statistics0.650.62

Our claim is approximate rather than perfect security: the learned model does not attain exact distributional equality but is in practice hard to detect.

Weakness 2: Reliability and effective payload rate. We agree that nominal rate does not account for decoding errors. We now quantify effective capacity and show reliable recovery through repetition.

Repetition: We think it is plausible that an attacker obtains multiple stegotexts encoding the same secret, for example because the same API key is reused across chats or because one long chat carries the secret several times. To combine multiple stegotexts into a single decoding, we sum the last-token belief log-probabilities of each block and take the argmax.

Full 128-bit recovery rate by number of repetitions (24 secrets x prompts):

RepsBaseInstruct
10.00080.0000
30.81690.2144
50.98880.7648
101.00000.9965

To compute the real payload capacity after discounting for decoding errors, we measured mutual information between the true and decoded digit, divided by tokens per block. That is an estimate of the rate an error-correcting code over many blocks could recover, subject to the relevant channel assumptions. We used 1000 sequences from the heldout dataset to measure this.

ModelTok/blockNominalMEC-LLM headMEC-LLM BayesGreedy MEC, logits recomputedGreedy MEC, encoder logits
1.5B80.500.320 (80.1%)0.369 (86.8%)0.108 (32.9%)0.350 (77.8%)
1.5B160.250.197 (87.4%)0.217 (93.0%)0.058 (35.1%)0.185 (81.3%)
3B80.500.339 (80.1%)0.408 (90.2%)0.141 (42.9%)0.341 (76.1%)

Values are bits/token, with block accuracy in brackets; 95% CIs are about ±0.002–0.007.

This estimated effective rate is distinct from a demonstrated finite-length ECC. The repetition experiment directly establishes that full-secret recovery can be made reliable, at the cost of additional tokens. The external greedy MEC comparison uses a practical implementation with finite-precision logit recomputation, not an assertion that our model outperforms ideal MEC.

We also optimized security + belief-head decoding accuracy directly, instead of explicitly optimizing coupling entropy:

Training objectiveDecoding accuracy ↑KL security ↓
KL-security + H(J) entropy + belief readout0.5650.295
KL-security + decoding accuracy0.8400.190

Weakness 3: Training objective and Bayesian posterior. We trained models with both objectives: the four-term coupling-entropy objective (Appendix E.1) and the Bayesian-posterior objective (Section 3.2–3.3). We will make explicit which checkpoint uses which loss. The mathematical KL identity is an evaluation identity for the resulting distributions; it does not require the model to have been trained with that particular loss. At each block boundary, the secret changes to a freshly and independently drawn 4-bit value. Its prior is therefore uniform, so resetting the current block's posterior is correct. This should not be confused with asserting that previous secrets cannot affect later hidden states, or that blockwise security automatically proves full-sequence security. Could the reviewer clarify whether their objection concerns resetting the prior for a newly sampled secret, or instead the claim of full-sequence security?

AuthorsOpenReview ↗

Response to Reviewer GkVU (2/2)

Question 1: What is the sequence-level security divergence of the evaluated checkpoint?

The mean per-token KL between the reference and MEC-LLM’s output distribution is 0.148 nats for the base model and 0.152 nats for the chat model.

Question 2: Was the released checkpoint trained with the objective of Section 3.2, and if not, do its security results apply?

Yes, the models were trained with the objective of section 3.2 and we confirmed security by evaluating checkpoints on the KL between reference and mec-llm’s output distributions and by various blackbox and greybox detection experiments outlined in section 4.4.

Question 3: How much payload survives at the reported error rate once an error-correcting code is applied?

To compute the real payload capacity after discounting for decoding errors, we measured mutual information between the true and decoded digit, divided by tokens per block. That is the rate an error-correcting code over many blocks can recover without errors. We used 1000 sequences from the heldout dataset to measure this.

The table below shows nominal capacity (bits per token without errors) and capacity after error-correction when decoding using MEC-LLM’s belief head and the external Bayesian decoder. To compare against the optimal solution, we also show capacity of the external greedy MEC encoder using its exact logits or recomputed. The latter is much lower because small numerical differences lead to much larger errors for the greedy decoder. Overall, capacity after error correction slightly surpasses the best known external encoder.

modeltok/blocknominalMEC-LLM headMEC-LLM Bayesgreedy MEC, logits recomputedgreedy MEC with the encoder's own logits
1.5B80.500.320 (80.1%)0.369 (86.8%)0.108 (32.9%)0.350 (77.8%)
1.5B160.250.197 (87.4%)0.217 (93.0%)0.058 (35.1%)0.185 (81.3%)
3B80.500.339 (80.1%)0.408 (90.2%)0.141 (42.9%)0.341 (76.1%)

Values are bits/token, with block accuracy in brackets; 95% CIs are about ±0.002–0.007.

Question 4: What context does the receiver need to decode, and does it include the secret-bearing prompt?

The receiver needs a similar context to help approximate the sender’s computed coupling. It does not need to include the secret. We don’t include the secret in any of our evals. We have added a sentence in our methodology section to clarify that the secret-bearing prompt is not needed for decoding.

We would like to kindly follow up as the discussion deadline is approaching. We have now uploaded the revised paper, with all changes highlighted in red for readability. We would greatly appreciate it if the reviewer could engage with our rebuttal and consider raising their score if our responses, additional experiments, and revisions have adequately addressed their concerns. Thank you for your time and consideration!

Thank you for the additional experiments and analyses. My concerns have been addressed, so I raise my score.

Reviewer_A5r8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether language models can internalize high-capacity steganography while preserving the output distribution of a reference model. The proposed MEC-LLM appends trainable layers to a frozen language model and jointly optimizes distribution matching and secret recovery using objectives motivated by minimum-entropy coupling. The authors also construct a model organism that extracts secrets from context and encodes them in assistant responses. On the defense side, the paper studies detection through the concentration of secret-conditioned output distributions, observable with logit access, and mitigation through paraphrasing to disrupt the covert channel.

Critical strengths
  • The method has good theoretical support, drawing on minimum-entropy coupling to unify distribution matching and secret decoding within a single training objective.
  • The paper investigates both logit-based detection and paraphrasing as defenses against the covert channel.
  • The paper provides a clear discussion of its limitations.
Critical weaknesses
  • The reported raw encoding rate does not account for decoding errors and therefore does not establish reliable payload transmission. This distinction is particularly relevant to the API-key exfiltration scenario, where exact recovery is necessary. Evaluating error correction and the resulting reliable transmission rate would be better.
  • The paraphrasing experiments demonstrate an effective intervention against the current construction, but do not establish robustness against encoders trained to withstand paraphrasing.
  • In the reported conversational evaluation, text-only classifiers can readily distinguish MEC-LLM outputs from cover text, and the decoder achieves no full-secret recovery.
Questions for authors
  • Can the authors quantify the reliably recoverable payload rate in the conversational setting?
  • How does MEC-LLM compare with a simpler learned encoder using the same architecture and training budget?
3 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer A5r8

We thank the reviewer for highlighting the “good theoretical support” and that we "investigate both logit-based detection and paraphrasing as defenses". Below, we will answer the reviewers critique and questions. We also updated the revised paper with these new results:

Weakness 1: Error-Correction

The reported raw encoding rate does not account for decoding errors. [...] exact recovery is necessary. Evaluating error correction [...] would be better.

Because our threat models require full internalization, an error-correcting code would itself have to live in the weights, which we do not think is straightforward. We instead explore repetition and training on accuracy to solve the effective-capacity problem:

Repetition: We think it is plausible that an attacker obtains multiple stegotexts encoding the same secret. For example, the same API key might be reused across chats in an agentic coding setting, or because one long chat carries the secret several times. To combine multiple stegotexts into a single decoding, we sum the last-token belief log-probabilities of each block and take the argmax. This works exceptionally well because incorrect decodes are low-confidence often with substantial belief on the correct secret.

Full 128-bit recovery rate by number of repetitions (24 secrets x prompts):

repsbaseinstruct
10.00080.0000
30.81690.2144
50.98880.7648
101.00000.9965

Training on accuracy for emergent error-correcting codes:

We tested a training change that acts as an implicit error-correcting code. Instead of the three loss terms (security + coupling entropy + belief), we trained on security + belief head decoding accuracy alone. Theoretically, MEC is still the optimal solution to this new loss function and empirically, we found that the model still learns MEC because the coupling entropy and belief KL steadily declines during training. However, since it is optimized for decode accuracy, it learns a more robust/error-correcting MEC and has significantly higher decode accuracy at comparable security:

training objectivedecoding accuracy ↑KL security ↓
KL-security + H(J) entropy + belief readout (control)0.5650.295
KL-security + decoding accuracy0.8400.190

Performance of MEC-LLM vs external baselines

Lastly, we want to highlight that the imperfect recovery is not a failure of MEC-LLM but caused by the limited amount of available entropy: Posttrained models have lower entropy and even a perfect encoder might not always be able to encode a message perfectly. To show that MEC-LLM is close to optimal, we compare its capacity after error-correction with the external greedy encoder (which is close to optimal). We find that MEC-LLM’s capacity is slightly higher and that it is more robust against low-precision noise: If logits are not bit-identical, greedy MEC’s performance drops by more than half but MEC-LLM is robust. We show our results in section Question 1.

Weakness 2: Paraphrasing against robust encoders

The paraphrasing experiments demonstrate an effective intervention against the current construction, but do not establish robustness against encoders trained to withstand paraphrasing.

We agree with the authors that our paraphrasing experiments don’t establish robustness against paraphrasing-resistant encoders. However, we don’t think this in scope or necessary for the paper because MEC-LLM can’t be readily trained to be paraphrasing resistant. We are also not aware of secure/high-capacity stegomodels that are paraphrasing-resistant. Ultimately, we wanted to test defenses against the model organism we developed, and that model organism is not paraphrasing resistant.

Weakness 3:

In the reported conversational evaluation, text-only classifiers can readily distinguish MEC-LLM outputs from cover text, and the decoder achieves no full-secret recovery.

We thank the author for pointing out that text-only classifiers in Table 10 readily detect MEC-LLM. We tracked the issue down and found that the chat model’s entropy is so low that even a perfect external greedy MEC encoder only achieved an accuracy of 77.8% and that MEC-LLM learned to trade security for decode accuracy. We retrained MEC-LLM-1.5B with 8 and 16 tokens per block. We find that the text-only classifiers perform much worse at 16 tokens per block. This suggests that the chat MEC-LLM is secure if it has enough tokens to encode the secret.

1.5B8 tokens/block16 tokens/block
text + PPL0.890.75
DistilBERT0.880.77
log PPL alone0.840.72
text stats0.650.62

For full-secret recovery, please see our answers to Weakness 1.

AuthorsOpenReview ↗

Rebuttal to Reviewer A5r8 (2/2)

Question 1: Payload Rate:

Can the authors quantify the reliably recoverable payload rate in the conversational setting?

The recoverable payload rate is constrained by the entropy of the output distribution as a ceiling on achievable capacity. Because LLMs that are larger and more post-trained have lower entropy, their capacity is smaller. This is true for both MEC-LLM and the mathematical optimal encoder.

The table below shows nominal capacity (bits per token without errors) and capacity after error-correction when decoding using MEC-LLM’s belief head and the external Bayesian decoder. To compare against the optimal solution, we also show the capacity of the external greedy MEC encoder using its exact logits or recomputed. The latter is much lower because small numerical differences lead to much larger errors for the greedy decoder. Overall, capacity after error correction slightly surpasses the best known external encoder.

modeltok/blocknominalMEC-LLM headMEC-LLM Bayesgreedy MEC, logits recomputedgreedy MEC with the encoder's own logits
1.5B80.500.320 (80.1%)0.369 (86.8%)0.108 (32.9%)0.350 (77.8%)
1.5B160.250.197 (87.4%)0.217 (93.0%)0.058 (35.1%)0.185 (81.3%)
3B80.500.339 (80.1%)0.408 (90.2%)0.141 (42.9%)0.341 (76.1%)

Values are bits/token, with block accuracy in brackets; 95% CIs are about ±0.002–0.007.

Question 2: Simpler Encoder:

How does MEC-LLM compare with a simpler learned encoder using the same architecture and training budget?

We agree that MEC-LLM should be compared to simple baselines. We did a literature review but unfortunately did not find existing internalization methods that are simple and secure. However, we believe that we already train with a loss that is simple and doesn’t explicitly train the model to learn to minimize the coupling entropy:

We train MEC-LLM with two different losses: (1) Minimizing the entropy of the coupling directly by optimizing for H(Coupling) + KL_security + KL_belief (Appendix E.1, page 15). (2) Minimizing belief crossentropy while keeping security up: KL_security + CE(Bayesian belief | model’s belief) as described in section 3.3.

We believe that loss (2) is simple and not constrained to MEC: It directly optimizes for decodability rather than explicitly minimizing the coupling entropy. Indeed, we found that training with loss (2) achieves better performance and we trained models with (2). For numerical results, please see our table in Weakness 1.

We also SFT’ed a model with completions generated with the external greedy encoder. However, performance never exceeded chance, suggesting that MEC can’t easily be learned from SFT-ing simple externally-generated rollouts. For details and controls, please see our answers to reviewer 6sVY20.

Thank you for your response. Most of my concerns have been addressed, and I will maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.