acceptodds
Under review as a conference paper at ICLR 2027

Plan Canvas: Fixed Reasoning Regions for Continuous Language Flows

Abstract

Continuous language flows generate text by denoising all positions of a target canvas together. The natural way to add reasoning to such a model is to write a trace ahead of the answer, but the trace length changes from question to question. The answer start is therefore unknown during denoising, and the model has to decide the trace length, the place of every trace token, and the answer at the same time. We propose Plan Canvas to fix the boundary between the trace and the answer. A plan region of fixed capacity holds a compact trace, supervised padding fills its unused positions, and the answer starts at a fixed position. The fixed regions also allow separate denoising clocks for the plan and for the answer. With the trace text, backbone, and canvas length of the free-trace baseline held fixed, Plan Canvas improves accuracy on ProsQA and on Deep ProsQA, a graph benchmark with longer proofs. On Deep ProsQA, accuracy rises from 73.0% to 87.0%, the share of questions answered with a valid path rises from 30.8% to 59.1%, and the gain is largest on the longest proofs.

Primary area:
generative models
Keywords:
continuous language flows, diffusion language models, flow matching, parallel text generation, reasoning traces, structured generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_v7kKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Plan Canvas proposes a fixed reasoning-region design for continuous language flows, aiming to mitigate the problem of answer positions being unfixed due to variations in reasoning text length. This method reserves a region with fixed capacity for reasoning text, fills unused positions with padding symbols, and further introduces segment clocks to separately control the denoising progress of the reasoning region and the answer region. The experimental results show that, on Deep ProsQA, the single-clock and segment-clock versions improve accuracy from 73.0% with free trace to 82.1% and 87.0%, respectively, while reducing repetition and invalid steps in the generation trajectory. This indicates that fixed reasoning regions can improve the performance of this model on the tested graph reasoning tasks.

Critical strengths

S1. The method achieves clear performance improvements on the tested tasks. Plan Canvas and free trace use the same backbone, compact reasoning text, and sequence-length budget, and include an answer-then-trace control. These results support the effectiveness of the method under the current experimental settings.

S2. Analysis of generation trajectories helps explain the performance improvements. In addition to reporting answer accuracy, the authors examine whether the generated trajectories are valid, whether they contain repetitions, and whether their lengths are appropriate. These analyses indicate that the improvements are also reflected in the quality of the generated reasoning trajectories.

Critical weaknesses

W1. Insufficient experimental coverage. All experiments are based on the same ELF-L backbone, and the tasks are limited to ProsQA and Deep ProsQA, a deeper variant constructed by the authors. Although the two datasets differ in reasoning depth, both are synthetic graph reasoning tasks with relatively regular structures. The current evidence is insufficient to establish whether the benefits of fixed reasoning regions can generalize to other backbones and to tasks with more irregular reasoning formats and lengths.

W2. Unclear stability of the results. Only one training result is reported for each configuration, and the existing results cannot establish whether the benefits of the method are stable across different random seeds. Furthermore, segment clocks reduce accuracy on ProsQA but improve accuracy on Deep ProsQA compared with the single-clock version, and the conditions under which this configuration is beneficial remain unclear.

Questions for authors
  1. Could the benefits of Plan Canvas over free trace be validated on another backbone, with the mean and variance across multiple independent training runs reported for the key configurations?

  2. Could at least one task with a reasoning format substantially different from ProsQA, such as natural language mathematical reasoning, be added to examine whether the method remains effective when reasoning lengths and expression formats are more irregular?

6 comments
AuthorsOpenReview ↗

Response to Reviewer v7kK [1/2]

W1. Insufficient experimental coverage. All experiments are based on the same ELF-L backbone, and the tasks are limited to ProsQA and Deep ProsQA, a deeper variant constructed by the authors. Although the two datasets differ in reasoning depth, both are synthetic graph reasoning tasks with relatively regular structures. The current evidence is insufficient to establish whether the benefits of fixed reasoning regions can generalize to other backbones and to tasks with more irregular reasoning formats and lengths.

Q2. Could at least one task with a reasoning format substantially different from ProsQA, such as natural language mathematical reasoning, be added to examine whether the method remains effective when reasoning lengths and expression formats are more irregular?

A1. Thank you for this suggestion. We agree that math is the natural next test, and we ran a new experiment for this rebuttal to answer it. The released ELF-L checkpoint was pretrained only on OpenWebText, which contains little mathematical text, so evaluating it on GSM8K directly would mostly measure what the backbone has not seen rather than the layout. To make the test fair to every layout, we first continued pretraining ELF-L for 10B tokens on Nemotron-Pretraining-Specialized-v1.1, sampling math, code, science, and other data at a 3:3:2:2 ratio. From this checkpoint we fine-tuned each layout on GSM8K under the protocol of the paper, with the same solution text, canvas length, and training budget for all layouts. The ordering from the graph tasks carries over to GSM8K: both Plan Canvas variants score above the free trace (8.0% and 10.4% vs. 7.4%), and the answer-then-trace control scores lowest (6.0%).

LayoutGSM8K accuracy (%)
Direct answer6.8
Free trace7.4
Answer then trace6.0
Plan Canvas, single clock8.0
Plan Canvas, segment clocks10.4

With the same solution text, the free trace adds 0.6 points over direct answering, and Plan Canvas with segment clocks adds 3.6 points. A fixed answer position alone does not help, since the answer-then-trace control falls below direct answering. As on Deep ProsQA, segment clocks add to the fixed layout (10.4% vs. 8.0%). The absolute accuracy is bounded by the scale of the 652M-parameter backbone, and the comparison of interest is between layouts on the same checkpoint.

On the graph tasks, Deep ProsQA balances proofs from 4 to 12 steps, so the trace length varies threefold within one dataset, and the gain over the free trace grows from +11.7 points on 4 to 7 steps to +15.9 points on 8 to 12 steps.

As for the backbone, our question is specific to continuous language flows, and ELF is the open continuous flow model available at this scale. Other public diffusion LLMs such as LLaDA and Dream are discrete masked models built around mask tokens and unmasking schedules. Because a continuous flow updates every target vector at every step, an unfixed trace boundary perturbs the entire trajectory, which is the cost Plan Canvas removes. Within ELF, we also evaluate two conditioning interfaces (single clock and segment clocks) and each intermediate component in Table 4, with Plan Canvas beating the free trace across all of them.

AuthorsOpenReview ↗

Response to Reviewer v7kK [2/2]

W2. Unclear stability of the results. Only one training result is reported for each configuration, and the existing results cannot establish whether the benefits of the method are stable across different random seeds. Furthermore, segment clocks reduce accuracy on ProsQA but improve accuracy on Deep ProsQA compared with the single-clock version, and the conditions under which this configuration is beneficial remain unclear.

Q1. Could the benefits of Plan Canvas over free trace be validated on another backbone, with the mean and variance across multiple independent training runs reported for the key configurations?

A2. Thank you. We retrained the key configurations with three seeds, including the run reported in the paper. Plan Canvas stays above the free trace on both benchmarks in the three-seed mean, with gaps of 5.2 and 13.8 points for segment clocks.

LayoutProsQA (paper)ProsQA (3-seed mean)Deep ProsQA (paper)Deep ProsQA (3-seed mean)
Free trace91.491.073.072.6
Plan Canvas, single clock96.896.382.181.5
Plan Canvas, segment clocks95.296.287.086.4

The backbone part of Q1 is addressed in A1 above. The means stay within one point of the single runs in the paper, and we will report the three-seed means in the revision. On ProsQA, the two clock interfaces are within 0.1 points of each other in the mean, since both are close to the ceiling. Segment clocks help when plans are longer, adding 4.9 points on Deep ProsQA in the mean, and the component ablation in Table 4 rises from 82.1% to 87.0% as each part is added. Longer plans gain more from a dedicated clock, so we use segment clocks as the default.

We hope the clarifications and additional details address the reviewer's concerns. Should any important concern remain, we would be grateful if the reviewer could bring it to our attention during the discussion period. In light of the additional evidence, we would appreciate it if the reviewer reconsiders their score.

Reviewer_t2WROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the use of a fixed template for reasoning in diffusion models. In their approach, a fixed length is allocated for reasoning traces and the answer always begins at a fixed location. This approach avoids problems that arise with not knowing the length of a reasoning trace. The fixed lengths for reasoning and the answer allows for separate clocks for denoising the reasoning and the answer. They show that their approach results in improvements on two tasks: ProsQA and on Deep ProsQA.

Critical strengths

The paper carries out experiments on two graph reasoning benchmarks and shows improvements.

The approach allows improvements on results using reasoning in a diffusion model, just as reasoning improved autoregressive approaches.

Critical weaknesses

The use of a fixed template for reasoning and answer in a diffusion framework is not new. Several papers have presented this approach previously. The authors don't cite either one (see references below). I don't see anything new over these previous papers.

While previous approaches using fixed templates provided additional benefits (e.g., early exits, reasoning with any order infilling), the authors do not provide additional benefits other than improving task performance.

The authors use a frozen T5-encoder which is a relatively old model.

See:

Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. Diffusion language models know the answer before decoding. arXiv preprint arXiv:2508.19982, 2025.

Raghav Singhal*, Zachary Horvitz*, Hao Zou, Carles Domingo-Enrich, Zhou Yu, Rajesh Ranganath, and Kathleen McKeown. Rethinking Reasoning with MDLMs: Early Exits, Post-hoc Reasoning, and Beyond. COLM 2026.

Questions for authors

If you do think there is something new over prior work which used fixed plans for reasoning and answers, provide details on what is new.

4 comments
AuthorsOpenReview ↗

Response to Reviewer t2WR [1/1]

W1. The use of a fixed template for reasoning and answer in a diffusion framework is not new. Several papers have presented this approach previously. The authors don't cite either one (see references below). I don't see anything new over these previous papers.

Q1. If you do think there is something new over prior work which used fixed plans for reasoning and answers, provide details on what is new.

A1. Thank you for both references. We will cite and discuss them. Both papers use a prefilled template on pretrained masked models to read or condition on the answer block, without testing whether a fixed boundary makes the same supervised trace more effective. Li et al. (2025) append an answer suffix to LLaDA and Dream for training-free early stopping once the answer region is confident. Singhal et al. (2026) prefill a delimiter between reasoning and answer blocks to bound answer entropy for early exits, sample reasoning given a known answer, and score partial traces. Their template gain is measured against prompting without a template (e.g., 13.9% to 51.2% for LLaDA-8B-Base on GSM8K), and Section 5.1 notes the fixed reasoning length as a potential drawback. Our matched comparison tests that fixed-length boundary directly by training all layouts on the same compact trace, backbone, and canvas length, varying only where the trace and answer sit. Fixing the plan region alone raises Deep ProsQA accuracy from 73.0% to 82.1% and valid paths from 30.8% to 55.0% (p<10−4p<10^{-4}). Meanwhile, our answer-then-trace control fixes the answer position while leaving the trace end open, and it repeats concepts in 28.5% of outputs, matching the free trace (26.8%) rather than Plan Canvas (9.7%). The gain thus comes from a bounded plan region whose end the model learns from supervised PAD, rather than from a fixed answer slot. Our setting is also a continuous language flow where every target vector starts from Gaussian noise and updates at every step without mask tokens, so the boundary must be learned during training instead of prefilled at inference. Those fixed regions then allow segment clocks with a separate denoising time per region, adding 4.9 points on Deep ProsQA (82.1% to 87.0%, p=4.5×10−5p=4.5\times10^{-5}).


W2. While previous approaches using fixed templates provided additional benefits (e.g., early exits, reasoning with any order infilling), the authors do not provide additional benefits other than improving task performance.

A2. Beyond accuracy, Plan Canvas makes the emitted proofs valid and turns the denoising order of plan and answer into an inference-time choice on a single checkpoint. On Deep ProsQA, valid paths rise from 30.8% to 59.1% and exact-length proofs from 40.6% to 70.0%, while repeated concepts drop from 26.8% to 9.7%. This matches the path validity of the full proof trace (59.1% vs. 58.5%) inside an 80-position readout window instead of 192, using the same canvas length and 129 backbone calls as the free trace. With segment clocks, the same trained weights can denoise plan and answer together, let the plan lead, or finish either region first (Table 5). Singhal et al. (2026) note that unconstrained any-order decoding in masked models often writes an answer early and fills in a post-hoc rationale afterward. Segment clocks let us test region order directly on fixed weights, where answer-first drops accuracy from 87.0% to 85.4% (p=0.023p=0.023) and a plan lead reaches 89.0% at 193 backbone calls.


W3. The authors use a frozen T5-encoder which is a relatively old model.

A3. We keep the frozen T5 encoder because it is part of ELF, the released continuous language flow we build on (Hu et al., 2026). ELF-L was pretrained on OpenWebText in T5's embedding space, so replacing the encoder would require pretraining a new flow backbone from scratch. In ELF, the encoder sets the embedding space and encodes the prompt, while the 652M-parameter flow Transformer runs every denoising update and decodes the final tokens. Because all compared layouts share the same encoder, initialization, and training budget, the encoder does not confound our layout comparison.

We hope the clarifications and additional details address the reviewer's concerns. Should any important concern remain, we would be grateful if the reviewer could bring it to our attention during the discussion period. In light of the additional evidence, we would appreciate it if the reviewer reconsiders their score.

Dear Reviewer t2WR Thank you once again for your valuable comments on our submission. As the discussion phase is approaching its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns. Should there be any remaining questions requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly.

AuthorsOpenReview ↗

A brief follow-up

Dear Reviewer t2WR, If you have a moment before the discussion closes, we’d be grateful to hear whether you have any remaining concerns. We’d be happy to clarify anything further. Thank you for your time and consideration! Best wishes.

Reviewer_SM2aOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Plan Canvas, a framework for reasoning in continuous diffusion language models that allocates fixed regions for reasoning traces and answers, avoiding the uncertainty associated with variable-length reasoning. By introducing supervised padding and separate denoising schedules, the approach improves answer accuracy (73% to 87%) and reasoning-path validity (30.8% to 59.1%) on Deep ProsQA.

Critical strengths

Empirical performance on the stated benchmarks exceeds baselines, suggesting the approach has merits.

Critical weaknesses

Not very transparent about limitations; would be interesting to understand in what scenarios (and what graph properties) does the proposed scheme not work (or if the fixed reasoning size has to be expanded).

There’s also limited discussion of why the proposed approach does work; since the reasoning pad region is fixed size, there is bound to be some compaction which results in detrimental performance.

Questions for authors

In general, I would appreciate a bit more intuition behind the decisions made to facilitate training and inference. The current writing seems to lack that, and makes it quite mechanical.

Are we reaching near saturation at the ProsQA task? The performance is quite high to begin with. Unclear if the performance gains are coming from the ease of the task, or the proposed approach.

From reading the paper, it is unclear how reasoning helps answer generation in the context considered by the authors. Could they clarify? This is perhaps something that I have missed (and is likely captured by the bidirectional attention and different scheduling). The experiments on denoising order are useful, but they don’t really establish whether the answer depends on the reasoning trace. In fact, Table 2 shows that a substantial fraction of questions are answered correctly despite having invalid reasoning paths

5 comments
AuthorsOpenReview ↗

Response to Reviewer SM2a [1/2]

W1. Not very transparent about limitations; would be interesting to understand in what scenarios (and what graph properties) does the proposed scheme not work (or if the fixed reasoning size has to be expanded).

A1. Thank you for the question. The key factor is plan capacity relative to trace length, which Figure 3(c) tests directly. By default, KK rounds the longest training trace up to a multiple of eight, so no training trace is cut. When KK is too small, truncation hurts accuracy. At K=32K=32, 30.3% of the training traces are truncated and Deep ProsQA accuracy falls from 87.0% to 80.6%. Over-allocating adds padding and costs less (85.5% at K=128K=128), and every capacity from 32 to 128 still beats the free trace (73.0%). For graph structure, Figure 3(a) splits Deep ProsQA by proof length from 4 to 12 steps, where the gain over the free trace grows from 11.7 points on 4 to 7 steps to 15.9 points on 8 to 12 steps. We will add a limitations paragraph stating this capacity rule and the truncation result.


W2. There's also limited discussion of why the proposed approach does work; since the reasoning pad region is fixed size, there is bound to be some compaction which results in detrimental performance.

A2. The fixed plan region does not compress or shorten the trace. Plan Canvas and the free trace supervise the exact same trace tokens on a canvas of the same total length (848 positions on Deep ProsQA, Table 1). Because the capacity covers every training trace, unused plan slots are filled with supervised PAD tokens and the answer always starts at a fixed position. In the paper, "compact" only refers to writing visited concept names instead of full proof sentences, a format shared across the compact-trace layouts. Keeping the full proof sentences (full trace) scores 73.5%, against 87.0% for Plan Canvas.


Q1. In general, I would appreciate a bit more intuition behind the decisions made to facilitate training and inference. The current writing seems to lack that, and makes it quite mechanical.

A3. Thank you. We will add this intuition to Section 2. In a free trace, the answer starts right after the last trace token, so during joint denoising the model must place every trace token without yet knowing where the trace ends. This moving boundary shows up in the generated paths, where 26.8% of free-trace outputs repeat earlier concepts (e.g., shimpus chepus shimpus chepus) and 51.8% run longer than the proof (Figure 2). Supervising PAD tokens in a fixed region teaches the model where the plan stops and detaches the answer start from trace length, cutting repeats to 9.7% and raising exact-length plans from 40.6% to 70.0%. Once every position belongs to a known region ahead of generation, each region can also take its own denoising time (segment clocks), allowing plan-first or answer-first sampling with one set of weights.


Q2. Are we reaching near saturation at the ProsQA task? The performance is quite high to begin with. Unclear if the performance gains are coming from the ease of the task, or the proposed approach.

A4. We built Deep ProsQA to test deeper proofs well below saturation. Every Deep ProsQA question pairs two chains of equal length so chain length cannot give away the answer, with proofs balanced from 4 to 12 steps. Direct answering scores 57.7% (near the 50% floor of a two-candidate question) and the free trace scores 73.0%, leaving plenty of headroom. Plan Canvas reaches 87.0% (+14.0 points, McNemar p<10−4p<10^{-4}), with its largest gain on the hardest 8 to 12 step proofs (+15.9 points). As proof depth increases, the margin over the free trace grows instead of shrinking.


AuthorsOpenReview ↗

Response to Reviewer SM2a [2/2]

Q3. From reading the paper, it is unclear how reasoning helps answer generation in the context considered by the authors. Could they clarify? This is perhaps something that I have missed (and is likely captured by the bidirectional attention and different scheduling). The experiments on denoising order are useful, but they don't really establish whether the answer depends on the reasoning trace. In fact, Table 2 shows that a substantial fraction of questions are answered correctly despite having invalid reasoning paths

A5. The answer attends bidirectionally to the plan in every joint update, and both ablations and schedule tests show that it relies on the plan. Under the same backbone and training setup, adding the plan region raises Deep ProsQA accuracy from 57.7% (direct answer) to 87.0%. Holding the trained weights fixed, denoising the answer before the plan lowers accuracy (85.4% vs. 87.0%, p=0.023p=0.023; 85.6% vs. 87.9% under mixed training, p=0.003p=0.003), whereas letting the plan lead keeps accuracy high (87.6% and 89.0%). Under Plan Canvas, every question with a valid path is also answered correctly (59.1% + 27.9% = 87.0% in Table 2). Correct answers with invalid paths arise because each question has two candidates (so a guess is right half the time) and because our path check invalidates the whole path for a single non-edge hop. Plan Canvas lowers this share from 42.5% (free trace) to 27.9%, so more correct answers carry a verified proof.

We hope the clarifications and additional details address the reviewer's concerns. Should any important concern remain, we would be grateful if the reviewer could bring it to our attention during the discussion period. In light of the additional evidence, we would appreciate it if the reviewer reconsiders their score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.