- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper builds on HRPO, which feeds both the sampled token embedding and a projection of the hidden state into the next reasoning step. DASH introduces a learned router that selects between token-only and hybrid inputs at each reasoning step, and a predictor that assigns a soft per-question budget. The budget is embedded into the reasoning inputs and used in the reward, but never truncates generation. Training uses questions and final-answer feedback, without CoT annotations, and proceeds through three RL stages: correctness, length bonuses for correct traces, and budget-aware optimization with predictor training. Qwen3-1.7B/4B/8B are trained on GSM8K and evaluated on GSM8K, GSM-Hard, SVAMP, MultiArith and MATH500. The authors report higher accuracy and shorter reasoning than SFT-CoT, and much shorter reasoning than HRPO at an accuracy cost. Validation includes end-to-end latency, routing and training-stage ablations, and fixed-versus-adaptive budget comparisons.
Critical strengths
-
The full training framework gives a useful accuracy–length trade-off. On Qwen3-4B, the four arithmetic benchmarks move from 72.96% accuracy with 99.00 reasoning tokens (SFT-CoT) to 78.19% with 36.72 tokens (DASH), and the reported latency includes the additional modules. The fixed B=96 variant in Appendix E.4 is particularly promising: 82.88% accuracy with 57.37 tokens, compared with 83.09% and 178.82 tokens for HRPO. This provides evidence that the budget-aware training framework can retain nearly all of HRPO's arithmetic accuracy at about a third of the reasoning length.
-
The supplementary evaluations strengthen the empirical evidence: fixed-routing and fixed-budget variants, stage removal, a question-level paired analysis, and routing comparisons across three training seeds. The authors also acknowledge important limitations, including that adaptive budgets do not establish a uniformly better frontier and that the straight-through routing gradient is biased. These controls and qualifications make the reported trade-offs easier to assess.
Critical weaknesses
-
The incremental benefits of the dynamic and adaptive mechanisms are not sufficiently compelling to establish them as substantial advances over HRPO. In Table 3, Fixed Hybrid retains the full curriculum and budget optimization but achieves 72.90% average accuracy, only 0.44 percentage points below DASH's 73.34%; Fixed Token-only reaches 72.71%. DASH also uses longer traces than both controls (50.19 versus 49.33 and 48.22 tokens). Appendix D.10 supports the repeatability of the small gains, but does not establish an improved trade-off at matched reasoning length or latency. Appendix F shows probability modulation around a mean Hybrid probability of about 0.6, but there is no state-independent stochastic mixture with a matched Hybrid rate to establish the value of conditioning the route on the reasoning state.
The adaptive-budget results raise a related concern about the value of input-dependent allocation. In Tables 16–17, fixed B=64 achieves 74.74% accuracy with 57.75 tokens, close to adaptive Bmax=96 at 74.53% with 55.20 tokens. This does not establish that fixed budgets dominate, and I acknowledge the favorable MATH500 result for adaptive Bmax=96. Nevertheless, the reported evidence mainly demonstrates alternative compression levels, rather than a clear and repeatable advantage from pairing budgets with individual questions. A uniformly superior frontier is not required, but a meaningful benefit at comparable cost is important to substantiate the paper's emphasis on dynamic routing and adaptive allocation.
-
The overall efficiency gains have not been evaluated against a competitive standard token-only RL baseline with a designed length penalty. The comparison with the original HRPO changes representation selection, rewards, budget conditioning and the curriculum simultaneously. Separate ablations retain much of the performance when routing or learned budget prediction is individually removed, but there is no simple token-only GRPO baseline whose length-penalty strength is tuned to trace out an accuracy–length curve. Fixed Token-only and Fixed Hybrid both retain the learned budget predictor and the full curriculum, so neither tests whether a simpler length-aware reward is sufficient. The supervision-based latent baselines also do not address this question. Without the comparison, it remains unclear how much of the practical improvement requires the proposed mechanisms rather than ordinary length-aware RL. This limits the significance of the contribution beyond the combined training framework.
Questions for authors
-
Routing: What evidence shows that dynamic routing improves over Fixed Hybrid or Fixed Token-only at matched reasoning length or latency? Would a state-independent stochastic router with the same Hybrid usage rate perform similarly? Latency measurements for the fixed-routing controls would help establish whether the small accuracy gains justify the additional mechanism.
-
Budget allocation: At comparable realized reasoning cost, what advantage does the learned budget predictor provide over tuned fixed budgets, and how stable is it across training seeds? Would shuffling predicted budgets across questions while preserving their distribution materially reduce performance?
-
Assessment-relevant comparison: How does DASH compare with token-only GRPO trained with a tuned length penalty, using the same backbone initialization and data with comparable training resources? Please report an accuracy–length curve. Applying the same penalty to HRPO would further clarify the gains beyond reward design.
-
Evaluation clarification: Could the authors report unadapted Qwen3 results under the same prompt, decoding and answer format, and separate formatting failures from answer errors? The similar SFT-w/o CoT GSM8K scores at 4B and 8B (27.07% and 27.35%) warrant clarification, although they do not establish a flawed baseline. Please also clarify how Bmax=64 was selected and whether test-set results informed that choice.
Thank you for your thoughtful review. We have addressed your concerns in our rebuttal and would appreciate any further feedback or clarification you may have. As the discussion period is nearing its end, we would be grateful if you could take a look at our responses when time permits.