- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents Hidden Tape, a procedure for encoding hidden messages in sequences of tokens used by LLMs. It exploits the fact that certain strings can be tokenized in two different ways to construct "bits", which can be concatenated together to convey multi-bit messages, and later decoded by reading off one bit at a time. The paper shows a small model can learn this string-to-bit mapping to encode and decode messages.
Critical strengths
None
Critical weaknesses
-
The significance is very minor. The paper uses an already known phenomenon (that a string can have two tokenizations) to encode and decode messages and trains LLMs to learn this simple, deterministic mapping. It is unclear what this encoding scheme could be practically used for and what scientific insight is gained by showing small LLMs can learn it.
-
The paper is incredibly hard to understand. It is filled with needless jargon that obfuscates the meaning of certain concepts, e.g., "tokenizer census", "experimental transport", "logical code", "canonical erasure", etc. The method lacks a clear progression from the idea (encoding bits of information in token sequences) to the implementation (using models to read and write them?) and the evaluation reads as a list of disconnected, poorly motivated experiments.
Questions for authors
What is the practical significance and scientific merit of this work?