- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes BRISK-DLM to improve verified parallel generation. The method combines correctness-filtered self-generated training, position weights based on decoding progress and forward-pass cost, and a lightweight prefix-conditioned corrector that reranks existing candidates. Experiments on 8B and 32B I-DLM report geometric-mean throughput gains of 22.7% and 29.8%, respectively, with additional ablations and a transfer study on DFlash.
Critical strengths
-
Clear motivation and coherent method design. The paper directly targets the early proposal errors that prevent later tokens from contributing decoding progress. Table 3 separates the effects of training data, position weighting, and the corrector, making the main design choices reasonably well supported.
-
Strong reported efficiency results with practical measurements. The evaluation covers two model scales, three workloads, and six concurrency levels. Throughput includes the corrector overhead, and the implementation study usefully distinguishes improvements in tokens per forward from actual wall-clock gains.
Critical weaknesses
-
Quality preservation is not uniform. At 32B, MATH-500 decreases from 97.6 to 96.0, MathBench from 95.6 to 92.8, and AIME-2024 from 83.3 to 76.7. Other tasks improve, but the paper does not provide uncertainty estimates. The results support a useful, task-dependent quality-efficiency trade-off rather than uniform quality preservation.
-
The specific advantage of the cost-aware weighting is not fully isolated. The controlled ablation compares dynamic and uniform weights, while the D-PACE comparison also changes the generation system. Comparisons with a fixed positional schedule or accepted-prefix weighting would make the contribution of the FPT-derived weights clearer.
Questions for authors
N/A
Thank you for recognizing our coherent design and practical efficiency gains. We appreciate your suggestions on quality uncertainty and more controlled weighting ablations.
W1. Task quality and evaluation variability.
To quantify evaluation variability, we report the mean and sample standard deviation across three evaluation runs for each model on seven benchmarks:
All results are mean ± sample standard deviation over three evaluation runs.
The repeated evaluations indicate stable performance across runs, with task-dependent changes rather than a uniform degradation across benchmarks. Fine-tuning updates the model parameters and can affect tasks differently; similar differences are also observed between Qwen3-32B and I-DLM-32B.
Across the seven benchmarks, the macro-average scores are 90.35% for Qwen3-32B, 90.64% for I-DLM-32B, and 89.98% for BRISK-DLM-32B. BRISK therefore maintains broadly comparable aggregate quality, with a 0.66 percentage-point decrease relative to I-DLM, while improving throughput by 29.8%.
W2. What does modeling forward-pass cost add to adaptive weighting?
Table 3 compares BRISK with standard uniform weighting. Following your suggestion, we also evaluated a fixed linear schedule that assigns the highest weight to the first proposal and decreases the weight linearly toward the last. Linear weighting performs comparably to BRISK early in training, when both emphasize the first few positions. As those positions become more reliable, BRISK shifts emphasis toward later tokens that can extend the accepted prefix, whereas the linear schedule continues to favor the same early positions.
On MATH-500 at stride N=8, with the same correctness-filtered training data and the corrector disabled, uniform weighting yields 15.55% rejection and 3.13 TPF, fixed linear weighting yields 15.20% rejection and 3.20 TPF, and BRISK achieves 14.03% rejection and 3.35 TPF. The lower test TPF of the linear schedule supports adapting positional priorities over training.