- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper trains a long-CoT model to split parts of its reasoning across parallel workers. A single model acts as both director and workers. SFT teaches the parallel format, followed by four stages of RL that reward correctness and penalize longest path length (LPL). An optional high length penalty stage further reduces LPL. On six math benchmarks, the method reduces LPL by about 35–40% relative to the original model, with mostly small changes in accuracy. Experiments on AIME 2024 also report lower wall-clock latency.
Critical strengths
-
The method gives useful accuracy–latency tradeoffs across several math benchmarks. The paper includes sequential baselines trained with a high length penalty. On AIME 2024, DC-CoT-HLP achieves 40.85% accuracy versus 39.03% for DSR-HLP-12K, with lower LPL (Table 1). Table 5 also reports lower actual latency, from 48.94 to 35.49 seconds. These results support a practical benefit in the tested setting.
-
The paper carefully handles the different contexts used for worker generation and director aggregation. The training masks and position IDs in Section 4.1.2 match these two operations, while inference uses standard vLLM calls. This is a useful implementation detail for training a shared model to perform both roles.
Critical weaknesses
-
The comparison does not isolate the benefit of the DC design from the additional training. The sequential HLP baselines receive 200 steps of further RL, while DC-CoT-HLP first receives SFT and 1,640 steps of RL before its 200-step HLP stage (Section 5.1 and Appendix I). Matching the final HLP stage is helpful, but does not control for this difference. Also, on AIME 2024, total output tokens fall from 9,540 for DSR-32K to 6,831 for DC-CoT, whose LPL is 5,974 (Tables 1 and 4). Much of the LPL reduction therefore comes with shorter outputs. A sequential baseline with comparable training is needed to establish the added value of the DC design.
-
LPL leaves out costs that matter for this architecture. It counts the decoding path but omits additional prefill and scheduling costs. Section 4.1.2 explains why worker generation and director aggregation require different context representations, so worker KV caches cannot simply all be reused unchanged. Tables 5–6 partly address this concern by reporting wall-clock time, but only on AIME 2024, without a breakdown of these costs. This limits how confidently the LPL gains across the other benchmarks can be interpreted as actual latency gains.
Questions for authors
-
With comparable SFT data and total training compute, does DC-CoT still improve the accuracy–latency tradeoff over a sequential model using the same RL and length-penalty schedule? A comparison on one existing benchmark would help separate the DC contribution from the effect of further training.
-
For the same trained DC-CoT checkpoint, how does actual latency change when workers run in parallel versus one after another, with their input contexts kept identical? Please break down prefill, decoding, and scheduling time, and state the prefix-cache reuse settings. This would measure the gain from parallel execution and the overhead omitted by LPL.