- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ConciseHint, which inserts conciseness instructions during reasoning generation, with length-dependent injection intervals and dynamically selected positions. ConciseHint-T learns hint embeddings from concise reasoning data and controls compression through embedding interpolation. Experiments across several models and benchmarks demonstrate substantial token savings and compatibility with existing efficiency methods, although accuracy preservation varies across settings.
Critical strengths
-
Substantial savings with a simple method. On Qwen3-4B, GSM8K token usage decreases from 2,381 to 1,213 while accuracy remains nearly unchanged. The manual variant requires no training.
-
Broad evaluation and practical compatibility. Experiments cover multiple models and tasks, and combining ConciseHint with existing methods frequently produces additional token savings. This supports its usefulness as an inference-time intervention.
Critical weaknesses
-
ConciseHint-T is evaluated only on Qwen3-1.7B, and stronger compression introduces accuracy losses. Without comparisons against tuned manual hints at comparable token usage, the results do not establish that learning improves the accuracy–efficiency tradeoff.
-
In Table 4, head insertion achieves higher accuracy (58.95% versus 55.56%) with fewer tokens (3,798 versus 3,880). Dynamic positioning could reduce recomputation, but a direct latency comparison is missing, leaving this central design choice insufficiently justified.
Questions for authors
In addition to the critical weaknesses, the following questions remain:
-
What are the complete ConciseHint-T training settings, including optimizer, learning rate, training duration, injection interval, and loss masking? The PDF provides no implementation link. Is code available to resolve these omissions?
-
Does the increasing-interval schedule outperform suitably tuned, larger fixed intervals at comparable token usage or injection counts? The current comparisons mainly demonstrate the harm of overly frequent intervention.
-
Can paired, question-level uncertainty estimates substantiate the accuracy-preservation claims, particularly on AIME24? Repeated sampling reduces decoding uncertainty but does not increase the number of distinct evaluation problems.
Response to reviewer JjXb
Dear Reviewer JjXb,
Thank you for your careful reading and valuable comments. We respond to your concerns point by point below.
1. W1. Does learning improve on a tuned manual hint?
Matched-token comparison. Our additional experiments demonstrate that learned hints outperform tuned manual hints under matched token budgets. We tuned the manual injection interval on 24 development questions and evaluated both methods on 192 held-out GSM8K questions under identical per-question token budgets. ConciseHint-T achieved 153 correct answers versus 141 for the tuned manual hint (+6.25 percentage points, paired 95% CI [1.56, 11.46]). This demonstrates the advantage of learned hints in producing correct answers under the same inference budgets.
Accuracy–efficiency trade-off and generalization. On GPQA, learned hints (γ=0.7) maintain identical accuracy (37.37%) with 16.2% fewer tokens than manual hints. On AIME24, γ=1 achieves higher accuracy than the original model (40.67% vs. 39.33%) while reducing token usage by 24.7%. The accuracy variation across γ reflects different compression–accuracy trade-offs. Importantly, learned hints are trained exclusively on GSM8K data, yet achieve substantial efficiency improvements on both AIME24 and GPQA, supporting their cross-domain effectiveness.
2. W2. Does dynamic position reduce latency?
Latency comparison between dynamic and head injection. Our additional experiments demonstrate that dynamic positioning reduces prefilling latency by 25% (148 → 111 ms per injection) compared with fixed head insertion on identical generated text. While head insertion achieves higher accuracy, it requires prefilling the entire generated chunk after each injection. Our dynamic strategy substantially reduces this recomputation overhead, providing a better balance between accuracy and computational cost.
3. Q1. How was ConciseHint-T trained, and is code available?
4. Q2. Can larger fixed intervals match the growing interval?
Larger fixed intervals. We have further compared our adaptive interval with Fixed-256/512/1024 (Qwen3-8B/GPQA-Diamond). All four settings achieved the same accuracy, while our adaptive interval reduced token usage by 13.0%/20.6%/35.7% and achieved lower end-to-end latency (88.2s vs. 103.1/114.5/148.0s), respectively. Matching six injections per question on average, a mixture of Fixed-256 and Fixed-512 used an estimated 525 more tokens per question than the adaptive interval.
5. Q3. How certain is accuracy preservation on AIME24?
AIME24 evaluation protocol and statistical significance. Since AIME24 contains only 30 questions, repeated sampling is a standard practice for obtaining reliable accuracy estimates, as also adopted by DeepSeek-R1 (64 samples per question). We follow this practice and further conduct paired question-level bootstrap to account for the limited number of distinct questions. For the NoWait comparison in Table 1, adding ConciseHint changes accuracy by only −0.67 percentage points on both Qwen3-4B (95% CI [−7.00, +6.00]) and Qwen3-8B (95% CI [−5.50, +4.00]). Neither difference is statistically significant, supporting our observation that ConciseHint largely preserves accuracy while improving reasoning efficiency.