- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether Llama-3.1-8B-Instruct forms an internal representation of response length before generating its first token. The authors conducted experiments including (1) hiding the length instruction from the model during generation after it has been read, (2) copying internal states from a prompt with a length instruction into one without it to find where length information is stored, (3) swapping the first few generated tokens between the two prompts, (4) extracting a low-dimensional “length direction” from internal states, and (5) adding this direction to new prompts that contain no length instruction. They find that length information is stored mainly at the chat-template tokens between the user message and the response, in the middle layers, and that changing these states before generation makes responses shorter or longer.
Critical strengths
- Each alternative explanation is tested directly. The paper lists three alternatives at the start (the model re-reads the instruction while generating, the instruction only changes the first tokens, or the internal states are correlated with length but do not cause it) and runs a separate experiment for each.
- The controls are well chosen. The extracted direction is compared with random directions of the same size, a direction outside the extracted space, and changing the full internal state. The effect also holds when the first 16 response tokens are fixed.
- The extraction method reduces the influence of topic and wording. The length direction is estimated only from changes within the same prompt, and only the part shared by two differently worded instruction sets is kept.
Critical weaknesses
W1. The scope of models, prompts, and settings is narrow.
- One model: All experiments use Llama-3.1-8B-Instruct. There are no other model families, no other sizes within the Llama family, and no base model. Since the question of whether language models plan before generating is general, a single 8B model is a significant limitation.
- One prompt source with no breakdown by task type: All prompts come from UltraChat, and results are not broken down by the kind of request (e.g., explanation, list, creative writing, coding). Natural response length varies strongly with task type, so it is unclear whether the effect holds uniformly or is driven by particular kinds of prompts.
- One decoding setting: All generations use greedy decoding. Whether the effect survives sampling, which is how models are typically used, is untested.
- One language and one chat template: The localization result depends on the specific template tokens of Llama-3.1, so it is unclear how it transfers to other templates.
The authors acknowledge these limitations, but they substantially restrict what the paper establishes.
W2. The results show that length information is stored before generation, but not that the model plans the length in advance.
The masking experiment hides only the tokens of the length instruction. The template positions (C1–C5) have already read the instruction during prefill, and the model attends to these positions again at every step of generation. The model may therefore simply store the requested length at C1–C5 and look it up while generating. This is different from deciding the length before generation begins, which is what the title and abstract suggest. The intervention experiment has the same issue: changing C5 changes what the model looks up during generation. To distinguish the two, the authors could restore the original C5 states after the first few generated tokens. If the change in length persists, the length was set early; if it disappears, the model relies on C5 throughout generation. Otherwise, the claim of a "plan" should be softened.
W3. Prior work on length steering is not discussed, and the new method is not compared with simpler ones.
Previous work has already shown that adding vectors to a model’s internal states can change response length or verbosity (Stolfo et al., 2024, "Improving Instruction-Following in Language Models through Activation Steering"; Azizi et al., "Activation Steering for Chain-of-Thought Compression"), but neither is cited. The main new contribution is therefore showing where and when length information forms, not that length can be steered, and the paper should state this clearly. In addition, the proposed extraction method is not clearly better than a simpler one. The authors should test whether a simpler direction, such as the difference between prompts asking for brief and detailed answers, has a similar effect when applied in the same way.
Questions for authors
Does a similar pre-generation length representation exist for reasoning? For example, with chain-of-thought prompting or models that produce a reasoning trace before the final answer, does the state at the template positions also predict and control the length of the reasoning, and does it separate the length of the reasoning from the length of the final answer?