acceptodds
Under review as a conference paper at ICLR 2027

Not All LLM Reasoning is Visible in the Chain-of-Thought

Abstract

A key question for AI safety is whether a language model expresses all of its reasoning in its output tokens. We demonstrate a concrete failure mode where frontier models exhibit invisible reasoning by leveraging semantically irrelevant filler tokens to improve performance on synthetic reasoning tasks. We evaluate 13 frontier language models across three tasks and find that many models benefit significantly from filler tokens, with accuracy improvements of up to 13 percentage points. The benefit depends on which tokens are used and differs across models. We further show that filler tokens enable Claude Opus 4.5 to satisfy a hidden modular arithmetic constraint without sacrificing accuracy on its primary task, demonstrating that invisible reasoning can serve objectives entirely invisible to CoT monitoring. Reinforcement learning gives Qwen3-235B strong preferences over filler token content, but neither RL nor supervised fine-tuning produces a filler token benefit that persists at test time. Our results indicate that frontier models already perform consequential computation with no interpretable trace in their output tokens.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
monitorability, safety, reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4FZLOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether language models can perform effective computation without interpretable CoT. The author evaluates fixed, task-agnostic "filler sequences" applied to tasks involving synthetic multiplication, arithmetic operations, and variable counting, examining how their effectiveness is influenced by token type, few-shot context and the model.

Critical strengths

The paper addresses a perspective previously unexplored in the literature, investigating the impact of filler words on performance across diverse models and tasks through a comprehensive set of experiments. It also reports on the effects and implications of training methods such as SFT and RL.

Critical weaknesses

While existing research has examined various "filling sequences," most analyses have been limited to performance comparisons at the sequence level, failing to fully elucidate the impact of factors such as semantics, regularity, tokenization structure, and prompt format. Consequently, no interpretable or predictable link has been established between the characteristics of the inserted content and model behavior; performance fluctuations are often simply attributed to an "opaque inference process." Although this paper demonstrates that the effects of filling sequences depend heavily on the model, task, and context, it fails to establish principles to explain or predict these variations. Thus, readers struggle to determine whether the results apply under untested conditions or how they might be leveraged to enhance inference. Regarding Chain-of-Thought (CoT) monitoring, experiments have shown that readable inference traces are absent for certain computations, yet the impact of this phenomenon on monitoring effectiveness has not been directly quantified. The paper needs to more clearly define its scientific contributions and scope of applicability.

Questions for authors
  1. I want to know how manys GPUs were used for the model training of Qwen3-235B (SFT and RL stage)

  2. Is there further evidence to distinguish the "task-specific computations" arising from the "fill position" from positional effects or attention-redistribution effects? In particular, how should the proposed diagnostic criteria be interpreted regarding the performance gains observed alongside "context-blocking"?

  3. For models that do not support the "assistant-prefill" function, please report the number of retained samples, the exclusion rates for both conditions, and the comparison results based on the shared subset of retained samples. How would the conclusions change if "Chain-of-Thought" (CoT) outputs were taken into account or treated as protocol violations?

Reviewer_pGFWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the effectiveness of fixed, problem-independent filler tokens to improve language models without a chain of thought. It examines 17 different types of fillers, various few-shot settings, and 13 models. The results show a significant dependence on the type of token, the task, the model, and the prompting context. The authors also examine why filler tokens help and whether they enable models to follow extra instructions without explicit reasoning. Finally, reinforcement learning makes the model prefer different filler tokens. However, after RL or SFT, using filler tokens does not consistently lead to better performance than simply answering directly with the same trained model.

Critical strengths
  1. The paper shows that the same filler sequence can have a negative effect on accuracy when the prompt includes no examples, but can improve accuracy when examples are present (Figure 1). This is useful because a particular filler sequence or prompt failing in one test does not mean that the model cannot still gain benefits from the use of fillers.

  2. The analysis provides useful clues about how filler tokens affect the model (Section 4.2). The authors found that restoring the early states recovers most of the accuracy gain, while restoring the late states does not. This indicates that better fillers affect the way the later layers process the problem, which is intuitive to understand.

Critical weaknesses
  1. The paper states that fillers may enhance accuracy, but it does not prove that this improvement is due to additional calculations taking place within the fillers. In Section 3, line 128, improved performance when using fillers is taken as evidence of invisible reasoning. Yet the inclusion of filler tokens can cause the model to produce different answers even if those tokens are not being used as a place to work out the solution. To support the stronger interpretation, the paper would need to provide evidence that the model carries out part of the calculation required for the current question in the filler states and then uses that calculation when giving its answer.

    Table 10 shows that an improvement in accuracy by itself is not sufficient. Accuracy is 57.3% when fillers are not used, 60.9% when they are allowed to use fillers, and 60.7% when fillers are prevented from accessing the question (Appendix C.1, lines 911–917; Table 10, lines 931–933). Under this condition, the filler states have no access to the numbers in the question and so are unable to calculate intermediate results from them. Yet almost the whole of the accuracy gain is still observed. To me, it is intuitive to expect the performance gains in language models with pause tokens.

    Although the other analyses do offer some helpful insights, they do not resolve this issue. Since the benefits of using fillers are already known, this uncertainty is important with regard to the contribution novelty. Jang et al. (2025) examine the effect of token choice and placement, while Brauer et al. (2026) investigate whether transferring specific problem information via filler states results in a change of answer. Neither of these studies is cited.

Questions for authors
  1. Do filler tokens help with intermediate calculations? Table 10 demonstrates that accuracy is still improved even when the filler states cannot access the question. What evidence from the other experiments indicates that the model carries out part of the calculation as it processes the filler tokens?

  2. What is the contribution of this paper to previous studies? What new insight do the results offer beyond those of Jang et al. (2025) and Brauer et al. (2026), in particular with respect to why fillers are helpful?

  3. Does Opus adhere to the extra rule without reducing the accuracy of multiplication? Table 2 gives the correct N/A responses for odd products. For even products, how frequently does Opus incorrectly give an N/A answer and how frequently does it provide the right numerical answer? Please include the results both with and without fillers, state the exact prompts used, and compare them with the results of ordinary multiplication for the same questions.

  4. Do results vary according to which outputs are discarded? Appendix A.4 excludes about 90 percent of the Opus 4.6 outputs since these include explicit reasoning. Please provide the number of outputs generated, the number discarded, and the number scored in each condition; the accuracy obtained on the questions that were retained in both conditions; and the percentage of all attempts that give a correct answer without any written reasoning.

Reviewer_qj5rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLMs gain accuracy from semantically irrelevant filler tokens when answering without a chain-of-thought, which the authors call invisible reasoning. A fixed filler sequence (17 types, e.g. counting numbers or ellipses) is prefilled before the answer, both in the test question and in the few-shot demonstrations. On three synthetic tasks (4-digit multiplication, nested arithmetic, variable counting in code), the authors sweep filler types and few-shot settings on Qwen3-235B, locate the effect with activation patching and probes, and evaluate 13 API models, several of which (e.g. Claude Opus 4.5, Gemini 3 Flash) gain 5-11 points on at least one task. Filler tokens also raise obedience to a hidden system-prompt constraint. RL and SFT on Qwen3-235B do not produce a test-time filler benefit, and RL with monitor rewards yields reward hacking rather than encoded reasoning.

Critical strengths
  1. Whether frontier models can compute in tokens that carry no information is relevant to CoT monitoring, and a systematic sweep over 17 filler types and 13 models with confidence intervals is more informative than the existing blog-post evidence.
  2. The authors are honest about negative results: RL and SFT do not transfer a filler benefit (Sec 6, Appendix F-G), and the limitations concede that the uplift is small relative to explicit CoT.
Critical weaknesses
  1. The headline finding is already known and the new evidence is mostly null. That recent frontier models gain no-CoT accuracy from filler tokens was reported by Greenblatt [1] on the same arithmetic task (Line 149), and the training-time version by Pfau et al. [2]. In Table 1, the deltas on the arithmetic task for GPT-5.5, Llama 4 Maverick, GLM-4.7 and Qwen3-235B are inside their 95% CIs, and the largest number, +30.0 for Opus 4.6, comes from a model where about 90% of samples were discarded for producing a CoT (Table 5). The finding should be stated as "a handful of models gain 5-11 points".
  2. Several claims outrun the evidence. The title and abstract talk about reasoning "invisible to CoT monitoring", but none of the filler-token experiments in Sec 4-5 has a CoT, and the "hidden" constraint in Sec 5.3 sits in the system prompt. Phenomenon 2 (performance depends on filler content) rests on the CV analysis of Table 3, where the std across types (0.5-1.2%) at 10-shot is comparable to the CI of a single point at N=1,000N=1{,}000 (about $\pm$1.4% in Table 1; Table 3 reports none), so the claimed "type inversion" is consistent with noise. A title like "Frontier models gain accuracy from filler tokens without chain-of-thought" would be accurate.
  3. Sec 6.2 replicates a known line of work without citing it. Roger & Greenblatt [3] introduced encoded reasoning under paraphrasing pressure, and Emmons et al. [4] already report that models struggle to hide reasoning that is necessary for the task, which is the conclusion of this section. The section itself only reports two numbers (96.5%, 94.4%) and qualitative descriptions of "obfuscation strategies".

[1] Greenblatt. Recent LLMs can use filler tokens or problem repeats to improve (no-CoT) math performance. Alignment Forum 2025.

[2] Pfau, Merrill, Bowman. Let's Think Dot by Dot: Hidden Computation in Transformer Language Models. COLM 2024.

[3] Roger, Greenblatt. Preventing Language Models From Hiding Their Reasoning. arXiv 2023.

[4] Emmons et al. When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors. arXiv 2025.

[5] Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. NeurIPS 2025.

Questions for authors
  1. Figure 1-3: What is the number of filler tokens nn? Table 3 shows the sign of the uplift changes with nn, so captions should state it.
  2. Line 179-180: The baseline also removes filler tokens from the demonstrations. Could you add a control with filler tokens in the demonstrations only?
  3. Line 258-259: Is the patch applied to all 94 layers at once or one layer at a time? How many examples were used for the probes in Table 9?
  4. Table 10 and 11 report different baselines for the same setting (10-shot 70.3% vs 64.0%). Why?
  5. Abstract: Where does "up to 13 percentage points" come from? The largest non-asterisked delta in Table 1 is +11.2.
  6. Sec 6.1: Collapsing to a few filler types after 30 RL steps looks like the usual diversity collapse of RL. What does it tell us about invisible reasoning?
  7. Appendix E uses clip-higher 0.28 and zero-advantage filtering, which are the tricks of DAPO [5]. Please cite it.
  8. Lanham et al. (2023) found no filler benefit for a 2023 model, and Pfau et al. [2] note that Claude 2 and GPT-3.5 gain nothing from filler tokens. Relating the uplift to model release date in Table 1 would make the contribution clearer.
  9. Typo:
    • Line 377: meets this criteria → meets this criterion
    • Table 2: 0-CoT → no-CoT
    • Table 1 Opus 4.6 arithmetic 91.7 vs Table 5 90.7: which is correct?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.