- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper aims to study the emergence of steganographic or encoded reasoning in chain of thought using an stylized experiment from Skaf et al (the coin flip one). Basically, this is a state-tracking task, where the prompt gives a starting side (Heads or Tails) and then 3–8 instructions. Each instruction is either “same” (the coin stays in the same state) or “different” (it changes to the other state). They then do GRPO on a few small models, where, the model is penalized whenever its reasoning contains the literal state words (Heads, Tails, Head, Tail, H, T). They found negative results, and some false positives, which they detail in the paper.
Critical strengths
- The paper picked a neat setting (coin flip) which they could then use to disambiguate similar-looking hypotheses about if the model is really stego or not. It is one among a few useful settings that has been used in other papers as well.
Critical weaknesses
- It is a negative result on some small, undertrained (AFAICT, 9,600 rollouts across 8 runs is 1,200 per run, so w 8 samples per prompt, that works out to about one prompt per step, or roughly 150 prompts per run), and older models. Trivially, stronger models can control their CoT (as per Yueh-Han et al. 2026; https://arxiv.org/abs/2603.05706) so it seems more like there were errors in their selection of model or their training setup. It seems much more likely, that the culprit to which induces stego in CoT traces is not an overt lexical penalty but one related to thinking length, which is a more well-known problem. As such the research contribution is quite small.
- The paper is very hard to read. It has basically no figures (the schematics are all also text!) and leaps into many engineering details that are irrelevant to the final scientific experiment or results. It also reports many things as if they are all of equal importance; it took quite awhile for the paper to get to the central experiment of interest.
Questions for authors
I do not have any questions whose answers could materially affect my assessment of the paper.