- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Plan Canvas proposes a fixed reasoning-region design for continuous language flows, aiming to mitigate the problem of answer positions being unfixed due to variations in reasoning text length. This method reserves a region with fixed capacity for reasoning text, fills unused positions with padding symbols, and further introduces segment clocks to separately control the denoising progress of the reasoning region and the answer region. The experimental results show that, on Deep ProsQA, the single-clock and segment-clock versions improve accuracy from 73.0% with free trace to 82.1% and 87.0%, respectively, while reducing repetition and invalid steps in the generation trajectory. This indicates that fixed reasoning regions can improve the performance of this model on the tested graph reasoning tasks.
Critical strengths
S1. The method achieves clear performance improvements on the tested tasks. Plan Canvas and free trace use the same backbone, compact reasoning text, and sequence-length budget, and include an answer-then-trace control. These results support the effectiveness of the method under the current experimental settings.
S2. Analysis of generation trajectories helps explain the performance improvements. In addition to reporting answer accuracy, the authors examine whether the generated trajectories are valid, whether they contain repetitions, and whether their lengths are appropriate. These analyses indicate that the improvements are also reflected in the quality of the generated reasoning trajectories.
Critical weaknesses
W1. Insufficient experimental coverage. All experiments are based on the same ELF-L backbone, and the tasks are limited to ProsQA and Deep ProsQA, a deeper variant constructed by the authors. Although the two datasets differ in reasoning depth, both are synthetic graph reasoning tasks with relatively regular structures. The current evidence is insufficient to establish whether the benefits of fixed reasoning regions can generalize to other backbones and to tasks with more irregular reasoning formats and lengths.
W2. Unclear stability of the results. Only one training result is reported for each configuration, and the existing results cannot establish whether the benefits of the method are stable across different random seeds. Furthermore, segment clocks reduce accuracy on ProsQA but improve accuracy on Deep ProsQA compared with the single-clock version, and the conditions under which this configuration is beneficial remain unclear.
Questions for authors
-
Could the benefits of Plan Canvas over free trace be validated on another backbone, with the mean and variance across multiple independent training runs reported for the key configurations?
-
Could at least one task with a reasoning format substantially different from ProsQA, such as natural language mathematical reasoning, be added to examine whether the method remains effective when reasoning lengths and expression formats are more irregular?
6 comments
Response to Reviewer v7kK [2/2]
W2. Unclear stability of the results. Only one training result is reported for each configuration, and the existing results cannot establish whether the benefits of the method are stable across different random seeds. Furthermore, segment clocks reduce accuracy on ProsQA but improve accuracy on Deep ProsQA compared with the single-clock version, and the conditions under which this configuration is beneficial remain unclear.
Q1. Could the benefits of Plan Canvas over free trace be validated on another backbone, with the mean and variance across multiple independent training runs reported for the key configurations?
A2. Thank you. We retrained the key configurations with three seeds, including the run reported in the paper. Plan Canvas stays above the free trace on both benchmarks in the three-seed mean, with gaps of 5.2 and 13.8 points for segment clocks.
| Layout | ProsQA (paper) | ProsQA (3-seed mean) | Deep ProsQA (paper) | Deep ProsQA (3-seed mean) |
|---|---|---|---|---|
| Free trace | 91.4 | 91.0 | 73.0 | 72.6 |
| Plan Canvas, single clock | 96.8 | 96.3 | 82.1 | 81.5 |
| Plan Canvas, segment clocks | 95.2 | 96.2 | 87.0 | 86.4 |
The backbone part of Q1 is addressed in A1 above. The means stay within one point of the single runs in the paper, and we will report the three-seed means in the revision. On ProsQA, the two clock interfaces are within 0.1 points of each other in the mean, since both are close to the ceiling. Segment clocks help when plans are longer, adding 4.9 points on Deep ProsQA in the mean, and the component ablation in Table 4 rises from 82.1% to 87.0% as each part is added. Longer plans gain more from a dedicated clock, so we use segment clocks as the default.
We hope the clarifications and additional details address the reviewer's concerns. Should any important concern remain, we would be grateful if the reviewer could bring it to our attention during the discussion period. In light of the additional evidence, we would appreciate it if the reviewer reconsiders their score.
Response to Reviewer v7kK [1/2]
W1. Insufficient experimental coverage. All experiments are based on the same ELF-L backbone, and the tasks are limited to ProsQA and Deep ProsQA, a deeper variant constructed by the authors. Although the two datasets differ in reasoning depth, both are synthetic graph reasoning tasks with relatively regular structures. The current evidence is insufficient to establish whether the benefits of fixed reasoning regions can generalize to other backbones and to tasks with more irregular reasoning formats and lengths.
Q2. Could at least one task with a reasoning format substantially different from ProsQA, such as natural language mathematical reasoning, be added to examine whether the method remains effective when reasoning lengths and expression formats are more irregular?
A1. Thank you for this suggestion. We agree that math is the natural next test, and we ran a new experiment for this rebuttal to answer it. The released ELF-L checkpoint was pretrained only on OpenWebText, which contains little mathematical text, so evaluating it on GSM8K directly would mostly measure what the backbone has not seen rather than the layout. To make the test fair to every layout, we first continued pretraining ELF-L for 10B tokens on Nemotron-Pretraining-Specialized-v1.1, sampling math, code, science, and other data at a 3:3:2:2 ratio. From this checkpoint we fine-tuned each layout on GSM8K under the protocol of the paper, with the same solution text, canvas length, and training budget for all layouts. The ordering from the graph tasks carries over to GSM8K: both Plan Canvas variants score above the free trace (8.0% and 10.4% vs. 7.4%), and the answer-then-trace control scores lowest (6.0%).
With the same solution text, the free trace adds 0.6 points over direct answering, and Plan Canvas with segment clocks adds 3.6 points. A fixed answer position alone does not help, since the answer-then-trace control falls below direct answering. As on Deep ProsQA, segment clocks add to the fixed layout (10.4% vs. 8.0%). The absolute accuracy is bounded by the scale of the 652M-parameter backbone, and the comparison of interest is between layouts on the same checkpoint.
On the graph tasks, Deep ProsQA balances proofs from 4 to 12 steps, so the trace length varies threefold within one dataset, and the gain over the free trace grows from +11.7 points on 4 to 7 steps to +15.9 points on 8 to 12 steps.
As for the backbone, our question is specific to continuous language flows, and ELF is the open continuous flow model available at this scale. Other public diffusion LLMs such as LLaDA and Dream are discrete masked models built around mask tokens and unmasking schedules. Because a continuous flow updates every target vector at every step, an unfixed trace boundary perturbs the entire trajectory, which is the cost Plan Canvas removes. Within ELF, we also evaluate two conditioning interfaces (single clock and segment clocks) and each intermediate component in Table 4, with Plan Canvas beating the free trace across all of them.