- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how much information about a private task input must enter a Transformer’s visible chain of thought (CoT), when the monitor sees the CoT but not the private input, private randomness, or final answer. For four task families and a fixed-precision Transformer model, it proves an information-theoretic footprint lower bound. For fixed depth and polynomially bounded model size and CoT length, successful exact computation requires near-linear input–trace mutual information, yielding an average-case CoT-length lower bound. A related entropy argument gives a qualified covert-computation length tax when the public input can be recovered from the trace. Finally, assuming suitable NC0 pseudorandom generators, a single-layer Transformer can encrypt intermediate computation online, making properties of the private input computationally inaccessible despite substantial mutual information. Validation includes three-seed LoRA experiments on two pretrained models and four synthetic tasks, prompting experiments across several model families, and separately trained one-layer encrypted-PARITY models. The latter reach 99.83% joint trajectory-and-answer accuracy at length 1,024 with substantial intermediate supervision. These results illustrate the mechanisms; they do not establish practical cryptographic security or spontaneous emergence of encrypted reasoning.
Critical strengths
- A substantive information-flow result with a useful conceptual separation. The footprint theorem strengthens an architectural requirement for intermediate computation into a lower bound on input–trace information, with an average-case CoT-length consequence. The subsequent encryption result shows why this information need not be available to a computationally bounded monitor. This directly sharpens what a CoT-monitoring guarantee would have to mean.
- An explicit architectural construction accompanied by informative controls. Online decryption, state update, and re-encryption are implemented within a single Transformer layer under a clearly identified cryptographic assumption. The empirical study includes matched Full-CoT controls, three-seed finetuning, payload recovery, negative results on Llama multiplication, and supervision ablations for encrypted PARITY. These controls make the constructive mechanism more convincing and help delimit its learnability.
Critical weaknesses
- The prompting study does not cleanly estimate unconditional covert-task success or the benefit of serial filler computation. Appendix D.1 reports accuracy only on nonleaking, appropriately parsed samples, while Table 11 shows that complete parsing failures can be large (for example, 61.3% for Qwen 3.5 0.8B on S5 Public-only CoT, averaged across difficulties). Calibration examples used to tune prompts are retained in the reported evaluation (D.1, pp. 46–47). Moreover, prompting fillers are appended to the user prompt, whereas finetuning fillers are generated autoregressively. This makes the prompting curves and scale fits less decisive as validation of the theoretical CoT regimes. Per-cell denominators, unconditional success rates, and results on the untouched post-calibration portion would materially improve the evidence.
- The practical monitoring implications need narrower wording. Corollary 3.2 assumes the public input is uniquely recoverable from the trace and compares against a minimal aligned construction; Figure 5 compares two prescribed protocols on differently correlated task pairs, so it illustrates the tradeoff rather than demonstrating a general measurable token penalty. The encrypted-PARITY experiment establishes execution of a supervised mechanism, while Appendix E explicitly reports one run, development use of test aggregates, failure of end-to-end-only training, and no monitor-attack evaluation. Together with the fact that an encrypted bitstring may be readily flagged as suspicious, the evidence supports limits on decoding private input properties, but not a general inability to detect hidden computation.
- The presentation needs a careful structural and copy-editing pass.The paper also has visible editing and layout issues: the results discussion contains a duplicated sentence (PDF p. 37); a result is referred to as “Theorem 3.2” where “Corollary 3.2” appears to be intended, alongside the phrase “extra extra tokens” (p. 7); and there are errors such as “an monitor,” “human-readbable,” “address this questions,” and “see alo” (pp. 1, 9, 9, and 27). Some figures are difficult to read at their presented size, Figure 4 appears before Figure 3, and the statement box for Corollary 3.2 breaks across pages.
Questions for authors
Q1 — Protocol clarification. Are all prompting fillers supplied in the user message, as D.1 and D.12 state, rather than emitted during autoregressive decoding? If so, please distinguish this input-padding intervention from the generated-Filler condition and qualify the shared Agnostic-CoT interpretation. A small generated-filler comparison would directly resolve this issue.
Q2 — Theorem scope. Can the main-text length-tax claim explicitly retain the requirement that the public input is recoverable from the trace and specify the asymptotic regime in which the bound actually separates the aligned and covert constructions? Please clarify what follows for equal-sized public/private tasks and ordinary nonminimal language-model traces.
Q3— Security and learnability scope. Please state alongside Figure 4 that finite-length cryptographic privacy was not tested, that each length uses a separate trained model, and that the apparent no-CoT threshold is partly tied to the 16-position preparatory supervision.
We thank the reviewer for their positive evaluation and would like to address their concerns.
We observed that smaller models (under 2B) had great difficulty in following the instructions properly, and resulted in leaking both questions in most of their responses, as highlighted in Appendix D.9. Most of the models still do not leak significantly, with the leakage contained within 15%. However, we saw in separate plots (Figures 31, 32, and 33) that the unconditional plots still follow the trends established by the conditional results. A similar fit is observed for both max-passing-d and the joint-correct accuracy in Figure 30.
This is in fact intentional: LMs do not produce filler CoTs by default. We thus deliberately put the prompting fillers in the user message so we could control content and length across models without relying on the models to generate the requested sequence. This follows prior input-padding work such as [1].
We also performed an experiment where we prompt Kimi K3 to produce fillers (Appendix E). Results are similar (Table 19) to those obtained when appending the fillers to the prompt; generating fillers does not improve accuracy compared to appending them.
Yes, prompting fillers are passed in the user message rather than emitted during autoregressive decoding. However, as per our testing, generated fillers do not significantly increase the joint-correct or private-only correct accuracies. They instead reduce the private-only accuracy when tested with Kimi K3 on multiplication. The experiments are outlined in Appendix E.
We have added the requested tables that refer to per-cell numerators and denominators that in Appendix D.11 as Tables 12, 13, 14, 15, 16, and 17 with the conditional accuracy and the numerator and denominator for their calculation. The unconditional plots have been added in Appendix F (figures 30, 31, 32, and 33), and, as highlighted above, follow the same trends as observed for the conditional case. For untouched, post-calibration accuracy figures, we observed the results to be very similar to our current setup. The accuracy deviations in both unconditional and conditional post-calibration-only settings are nominal, as observed in Figure 35. We see similar trends for post-calibration-only samples in Figure 34 for max-passing-d as we saw in Figures 3 and 30.
We agree. We have expanded the caption of Figure 5 to carefully make this point.
Yes, indeed, we do not mean to claim that the presence of covert computation can itself be hidden -- only its content. We are happy to highlight this more if the reviewer thinks that would be good.
Thank you for the careful reading. We have fixed most of them, and will take a careful pass to fix any remaining copy-editing issues. Regarding the order of Figure 3 and 4, LaTeX rendering does this to optimize the layout and we are inclined to keep it as is.
We address this as follows. First, we have now mentioned the requirement that the public input is recoverable from the trace in the text. Second, regarding the asymptotic regimes, we write "there is a substantial unavoidable length tax when Ncovert≫Npublic"; e.g., in the asymptotic where Ncovert=Npublic2, Eqs. 4 ≫ Eq 5 . We also do explicity state that an asymptotic separation is only claimed compared to a minimal-length aligned model ("minimally have to", line 205). Let us know if you think any further changes to the phrasing would help.
We have rephrased to acknowledge these points: we mention that each length uses a separate trained model, that the threshold may relate to the supervision signal, and that the claim about conjectural security is only in the N→∞ asymptotic.
[1] Brauer et al. Reading between the dots: Decoding hidden computation across filler tokens, 2026