acceptodds
Desk rejected at ICLR 2027

DO REASONING MODELS EVEN START? DECOMPOSING COT CONTROLLABILITY

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

The headline metric for chain-of-thought (CoT) controllability—the fraction of reasoning traces that comply *perfectly* with an instruction—conflates three quantities and is blind to one of them: how well a model *maintains* a constraint once it has started. Low CoT controllability is a pillar of the safety case for CoT monitoring, so *why* it is low decides whether that case is robust or one design choice away from failing. We grade 272k reasoning traces with position-resolved graders and treat the first violation as a time-to-event outcome, separating onset (does the model ever adopt the constraint), maintenance (slips per 1,000 words once adopted) and exposure (trace length), and we intervene on onset by opening the reasoning with a short compliant prefix. Across six open-weight models from three families (Qwen3, OLMo-3-Think, gpt-oss), the uppercase and alternating-case constraints are never started in five: every uppercase trace fails at the first word, and constrained traces carry the style no more often than unconstrained ones; gpt-oss-120b starts uppercase in half of its traces and drops it within a few words, and the two gpt-oss models start the required opening string but rarely close with it. Keyword suppression, in contrast, has no onset failure and leaks from the first word at a rate set by how often the question calls for the word. The benchmark score for "reason in uppercase" is 0% for every model we test, yet after a 20-word compliant prefix the slip rates of Qwen3-8B/32B and OLMo-3-7B/32B span 13× (95% CI 10–17), improve with scale within two of three families, and Qwen3-32B keeps an entire trace in uppercase 38–45% of the time, largely by continuing its context rather than by following the instruction.

Primary area:
interpretability and explainable AI
Keywords:
COT CONTROLLABILITY

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_igk8OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates why reasoning models fail to follow formatting and lexical constraints on their thinking traces (e.g., using only uppercase letters or avoiding banned keywords) on the CoT-Control benchmark. The authors argue that a binary pass/fail score across an entire trace obscures whether models fail at the very beginning, revert to normal text midway, or fail simply due to generating longer traces.

First, the authors modify the benchmark graders to locate the exact word or sentence of the first violation in each thinking trace. Using this violation position, they apply survival analysis and Shapley attribution to separately measure compliance over the first 20 words, subsequent violation rates per 1,000 words, and the effect of trace length. Second, they prefill the beginning of the thinking block with 20 or 100 words from the model's unconstrained trace rewritten into the target style. They show that models with 0% benchmark scores can sustain the style when given a compliant opening, and extend this analysis across model sizes, post-training checkpoints, and thinking versus final-output channels

Critical strengths
  • Practical relevance. The problem of understanding why reasoning models fail to follow constraints on their thinking traces—and distinguishing immediate failure at the first word from later reversion—is practical and relevant to chain-of-thought evaluation.

  • Extensive empirical evaluation. The paper includes extensive experiments across multiple model families, model scales, and post-training checkpoints, along with useful controls such as evaluating prefilled prefixes without the constraint instruction.

Critical weaknesses
  • Extremely poor clarity and organization. The paper is exceptionally difficult to follow due to overly compressed prose, constant cross-references to appendix tables, and inconsistent definitions of "onset" across sections (switching between 20-word survival, first-word compliance, and fitted mixture parameters).

  • Unclear central contribution. The paper combines survival analysis, gamma-frailty fitting, Shapley attribution, prefix prefilling, prompt hinting, checkpoint comparisons, and reasoning-effort ablations without establishing a coherent central narrative.

  • Limited novelty and conceptual contribution. The proposed framework primarily applies standard statistical tools to an existing benchmark without introducing substantial methodological novelty or creating new benchmarks. Moreover, the finding that models continue generating in uppercase after an uppercase prefix (even without instructions) largely reflects basic in-context style continuation rather than a new insight into reasoning controllability.

  • Interpretation of the prefilling intervention. Prefilling 20 or 100 words does not isolate "onset alone": it alters the conditioning context, selects different surviving subpopulations, and still significantly changes subsequent violation rates in several settings.

Questions for authors

Please refer to the weaknesses

1 comment

Thank you for the review. We take the clarity criticism seriously, since the other two reviewers raised it too.

Clarity and definitions. You are right that "onset" appears under three definitions. The revision will use one (first native unit complies) throughout, call the 20-word quantity "early survival", and keep the mixture fit only as an appendix check. We will also cut the appendix cross-references in the main text and bring the few numbers each claim relies on into it.

Central contribution. In one sentence: a 0% controllability score can mean two different things, "the model cannot keep the constraint" or "the model never starts", and these have different safety implications. We show how to tell them apart with a cheap diagnostic. The revision will be organised around that, with the other analyses in the appendix as support.

Novelty and style continuation. We agree that continuing an uppercase context is not surprising in itself, and the paper should say so more plainly. The no-instruction control exists exactly to measure it. What we think is new is what the control reveals: the share due to continuation varies widely (almost none in Qwen3-8B, most of the effect in Qwen3-32B), the OLMo-3 checkpoints with long-CoT SFT have lost it, and none of this is visible in a benchmark score that is 0% for all of them. On tools: the statistics are standard on purpose. The contribution is the measurement question, plus two concrete problems in the released benchmark (the "meow" loops scored as compliant, and the case-sensitive tag parser).

Prefilling does not isolate onset. Agreed, and we will stop claiming it does. The prefix changes the context and changes who survives. We will describe it as a probe that fixes the start so that what follows can be measured, report the cells where maintenance moves, and note that selection among adopters works against the contrasts we report, not in favour of them.

Reviewer_iXvoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that the CoT-Control benchmark (Chen et al., 2026) uses a binary trace-level score (a trace passes only if it fully complies with the constraint) that conflates three quantities: Onset (π): whether the model adopts the constraint at all. Maintenance (λ): the slip rate per 1,000 words among adopters. Exposure: trace length.

The authors build position-resolved graders and treat the first violation as a right-censored time-to-event outcome. They use Kaplan–Meier estimators, frailty models and a Shapley attribution of score gaps. They also intervene on onset by prefilling the reasoning with a 20- or 100-word compliant prefix, with a no-instruction control to separate instruction following from context continuation. They study six open-weight models from three families (Qwen3, OLMo-3-Think, gpt-oss), plus extra sizes and OLMo-3-7B checkpoints. The main claims are: Uppercase, alternating-case and opening-string constraints are never started (π = 0). The habitual “Okay,” opener already violates them. Keyword suppression has no onset failure and leaks from the first word at a rate set by how often the question calls for the word. gpt-oss-120b (and OLMo-3-7B for lowercase) starts and loses the style within a few words. With the plain score at 0% for every model, post-prefix uppercase slip rates differ by about 13× and improve with scale within Qwen3 and OLMo-3 (not gpt-oss). In OLMo-3-7B, maintenance is lost at the long-CoT SFT stage. The size and channel gaps reported by Chen et al. are re-attributed to exposure, maintenance or onset depending on the constraint.

Critical strengths
  1. Showing that a 0% score can mean “the first token is ‘Okay,’” is a clear and practically important point. It bears directly on how the CoT-Control number is used in the monitorability safety case.

  2. The position-resolved graders are fuzz-tested against the released graders (6,000 synthetic traces, all eight constraints). Censoring is handled explicitly, and bootstrap intervals are question-clustered with paired draws. The no-instruction control is the most informative design choice in the paper. It shows that Qwen3-8B’s gain is instruction following while Qwen3-32B’s is mostly continuation.

  3. The “meow meow …” loops with no content words are scored as compliant, which accounts for Qwen3-8B’s best plain score. This is a verifiable, actionable finding.

Critical weaknesses
  1. The operational split between “onset” and “maintenance” is not clean. π is defined as survival through the first w = 20 words. For suppression, “meow” and OLMo-3-7B lowercase, the authors acknowledge that π is early survival, not adoption. The “meow” attribution moves heavily with the window. In Table 25, onset is +7.2 at w = 20 but +0.3 at w = 1 for 8B → 32B. In the prefix experiments the clock restarts at the continuation, so π(w = 20) after a prefix is partly early maintenance. This sits uneasily with the claim that the prefix “moves onset alone.” The §3 regime assignments depend on thresholds chosen post hoc (F < 0.05, R < 0.25). The paper notes gaps around them, but the regimes are still descriptive categories, not tested hypotheses.

  2. Evidence that onset and maintenance “can be moved separately” is weak. The key comparison relies on Qwen3 lowercase, with about 190–200 plain-run adopters per model. At 8B the prefix/plain λ ratio is 1.30 [0.95, 1.74]. At 32B, K = 100 halves λ (0.43 [0.30, 0.60]), and the common-window comparison at K = 20 (0.61 [0.41, 0.95]) also excludes 1. The authors admit there is no pre-specified equivalence margin. The argument that “≤ 1.74-fold is small next to 7–8-fold” is heuristic. The pre-registered prediction that λ is independent of K was refuted at 8B. The accurate claim is “approximately but not independently separable,” and the abstract and contribution list should say so.

  3. The headline 13× contrast is between the two extreme models, OLMo-3-7B and Qwen3-32B, which differ in family, data, recipe, tokenizer and token budget. “Improves with scale” rests on two points per family outside Qwen3, with different recipes, and fails in gpt-oss. OLMo-3 runs use a 4,096-token budget against median traces of about 2,500 words, so many are truncated. The 12k-token rerun is reassuring but covers only 7B. Tokenization is handled in App. Table 21 (15× per 1,000 tokens), which is good, but this should be in the main text.

Questions for authors
  1. Can you separate onset from early maintenance after a prefix, for example with a definition independent of the window, and re-state the §4 claims under it?
  2. Do the maintenance rankings (and the “13×”) hold without prefill, using only the prompt-based hint or demo arms or a light fine-tune?
  3. Do the main Qwen3 size trends and the OLMo/Qwen contrast survive when only the masked grader is used?
  4. Can you apply the zero-cost diagnostics (first-word onset and π, no prefill required) to at least one frontier or API model, to show whether the regimes extend beyond ≤120B open models?
1 comment

Thank you for the detailed and fair review. We agree with most of the weaknesses and think they are fixable.

Onset vs. early maintenance (Q1). We agree the 20-word window mixes the two, especially after a prefix. We will switch to a window-free definition: onset is whether the first native unit complies (the "first word ok" column we already report), and everything after that is maintenance. Under this definition the §4 claim becomes simpler and more honest: the prefix takes first-unit compliance from 0 to near 1, and what differs across models is how quickly they slip afterwards. The 20-word quantity will be reported as early survival, not onset. We will also label the regimes as descriptive categories, since the thresholds were chosen after seeing the data.

Separability. We accept your wording. The abstract and contributions will say "approximately but not independently separable", with the 8B refutation and the 32B K=100 effect stated up front rather than in the appendix.

The 13× contrast and scale (W3). Fair. We will lead with the within-family ratios (about 5× in Qwen3 across four sizes, and in OLMo-3 across two) and present 13× as the cross-family range, with the per-token version (15×) moved to the main text. On truncation: the comparison uses words 20–100 after the prefix, which every trace reaches well before the 4,096-token budget, so the budget does not affect that number. We will extend the 12k-token rerun to OLMo-3-32B.

Without prefill (Q2). Only partly, and we should be upfront about that. Without a prefix, most models have no adopters for uppercase, so there is nothing to rank. Where the hint or demo prompt does produce a start, the result matches the prefix ordering: gpt-oss follows the hint and drops the style within a sentence, and Qwen3-8B with the hint keeps lowercase to the end in 29% of traces. We will add a table of post-start hazards under the hint and demo arms for every cell with enough adopters. We will list a light fine-tune as future work rather than promise it.

Masked grader only (Q3). Yes. With the masked grader the 20–100 hazards are 15.3 → 1.3 for Qwen3-8B → 32B and 41.0 → 7.3 for OLMo-3-7B → 32B, so both within-family trends hold and the cross-family span widens (about 31× vs. 13×). The strict grader is the conservative one. We will add masked values for 4B and 14B so the full size series is shown under both graders.

Frontier models (Q4). We would like this too. The difficulty is that the first-word diagnostic needs the raw reasoning, and most APIs return either a summary or nothing. We will apply the no-prefill diagnostics (first-word onset and early survival) to any frontier transcripts with raw CoT that we can obtain, starting with the released CoT-Control data, and otherwise state clearly that our regimes are established only for open models up to 120B.

Reviewer_TkZzOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why reasoning models fail all-or-nothing chain-of-thought constraints. It extends eight graders to locate the first violation and separates early adoption, subsequent first-slip hazard, and trace length using survival analysis. Across 271,500 traces covering three model families and additional sizes/training checkpoints, it finds failure regimes that binary scores obscure. Compliant prefixes reveal substantial differences in style maintenance, with no-instruction controls separating contextual continuation from instruction effects. A Shapley analysis revisits model-size and reasoning-channel score gaps.

Critical strengths
  1. The framework reveals meaningful differences hidden by binary compliance scores. By separating initial adoption, subsequent maintenance, and trace length, the paper explains why models can receive similar failure scores despite substantially different behavior. The prefix experiments reveal an ability to maintain constraints under suitable conditions that standard evaluations miss.

  2. The empirical evidence is thorough and transparent. The paper tests alternative explanations through repeated runs, grading checks, and controls for instructions and observation length. It also reports failed predictions and acknowledges that style maintenance has not been shown to predict monitor evasion. These checks strengthen confidence in the central findings.

Critical weaknesses
  1. The training-stage comparison does not fully isolate the cause of reduced maintenance. The checkpoints are evaluated with different task templates, prefix texts, and trace-selection rules. These differences make it difficult to attribute the decline specifically to imitation of distilled reasoning. A comparison under matched evaluation conditions, or a more cautious interpretation, would strengthen this claim.

  2. The score decomposition needs a more cautious interpretation. In one suppression channel comparison, the estimated contributions do not fully reproduce the observed score gap. Although the paper acknowledges this mismatch, the discussion presents the contribution of trace length too precisely. Reporting the residual alongside the attribution would clarify its limitations without changing the main descriptive findings.

Questions for authors
  1. Training-stage clarification / targeted control: Is there a base-versus-SFT or RL-Zero-versus-Think comparison using identical task formatting, prefix text, and trace inclusion rules? If unavailable, which training conclusions remain justified as associations?

  2. Attribution clarification: How does the mismatch between the fitted and observed suppression channel gaps affect the estimated contribution of exposure? Can Figure 4 explicitly distinguish the fitted and observed gaps?

  3. Definition clarification: Could you resolve the conflicting adoption values for Qwen3-4B external uppercase in Table 8 and clarify which short-trace convention is used in the downstream analysis?

1 comment

Thank you for the encouraging and precise review. All three questions point at real issues.

Training-stage comparison. You are right that the base → SFT link is not matched. Two parts of the trajectory already are: base and the RL-Zero checkpoints share one protocol (plain template, same prefix text, same trace rules), and SFT, DPO and Think share the chat protocol. Within each group the picture is clear: base and RL-Zero keep the style (uppercase hazard 3.8–5.5 per 1,000 words), while SFT, DPO and Think lose it (36–47). What is not matched is the comparison across the two groups. To close this we have rerun the SFT and Think checkpoints under the base-model protocol, and we will add these as matched rows in the revision. Until those are in, we will state the claim as an association: maintenance is low in every checkpoint that went through long-CoT SFT and high in every one that did not. We will drop "isolates the responsible step".

Attribution residual. Agreed. For the 8B suppression channel gap, the fitted gap is 32.7 pp against 27.8 pp observed, so 4.9 pp is unexplained. That residual is about a fifth of the exposure term (+22 [19, 25]), so the exposure estimate should be read as "the largest component", not as a precise number. At 14B and 32B the residual is under 0.5 pp. We will add the fitted gap, observed gap and residual to the attribution figure and table, and soften the wording at 8B.

Qwen3-4B external uppercase. Thank you for catching this, it is a bug on our side. The caption gives the correct value under our stated definition (0.19; a trace that ends compliant before word 20 counts as adopted). The table cell (0.09) was produced by an older script that dropped those short traces. The attribution and all downstream analysis use the stated definition. The two conventions only differ noticeably in the external channel and the low-effort gpt-oss cells, where traces are very short. We will regenerate the table so every cell uses the one definition, and the 4B external cells remain excluded from the attribution.

Comments on the paper

1 comment

To all reviewers

Thank you all for the careful reading. Two points came up in every review, and we agree with both.

The paper is too dense. We have tightened the prose throughout and shortened the appendix by five pages. The paper asks one question: does a 0% score mean "cannot" or "does not start"? The other analyses are there to support it.

"The prefix moves onset alone" is too strong. You are right: a prefix also changes the context, and maintenance does move in some cells (§4). We see the prefix as a probe that fixes the start so that what follows can be measured, not as a clean intervention on one parameter.

Thanks again for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.