- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work proposes Logic Density Pruning (LDP), a method that reduces redundant reasoning in LLMs by assessing the importance of complete reasoning steps. LDP combines a kernel-based dependence score and an attention-based citation score for step evaluation, employs adaptive thresholding for pruning, and uses a Step Perturbation Check to safeguard potentially important steps. Evaluations on three benchmarks and four reasoning models across commonsense, strategic, and mathematical reasoning tasks demonstrate its effectiveness.
Critical strengths
-
LDP evaluates the importance of complete reasoning steps by combining statistical dependence and attention-based evidence, with a step perturbation check to safeguard potentially important steps. This step-level design offers a principled alternative to token-level pruning and provides a practical means of reducing redundant reasoning while retaining answer-relevant information.
-
This work evaluates LDP on three benchmarks spanning commonsense, strategic, and mathematical reasoning, using four reasoning models. The results demonstrate substantial reductions in reasoning length across multiple settings, often with relatively small accuracy degradation. This breadth of evaluation supports the practical relevance and generalizability of the proposed approach.
Critical weaknesses
-
LDP combines a kernel-based dependence score with attention-based evidence and a hidden-state perturbation check. However, statistical dependence and attention weights do not necessarily reflect a step's causal contribution to answer correctness, while hidden-state changes do not directly establish whether removing a step harms reasoning. This limits confidence in the reliability of the pruning mechanism and the strength of the paper's central methodological claims.
-
Although the experiments cover three benchmarks and four models, the results show varying accuracy changes across settings, including noticeable drops in some cases. Moreover, reductions in reasoning-token counts do not by themselves establish proportional improvements in end-to-end latency or computational cost. More comprehensive evaluation of answer reliability, inference efficiency, and robustness would strengthen the case for practical adoption.
If the authors address my concerns, I'll raise my score.
Questions for authors
Could the authors provide evidence that the Step Perturbation Check reliably identifies reasoning steps whose removal would affect answer correctness?