- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies salience bias in LLM reasoning: models can get distracted by explicit numerical or procedural details and overlook basic commonsense constraints that make the task infeasible in the first place. The authors introduce SaliTrap, a benchmark with 1,145 examples covering four types of traps, and evaluate 12 LLMs, finding that this failure is widespread and becomes more severe as the amount of distracting numerical information increases. They further distinguish different failure modes, such as completely missing the trap, noticing it only after starting the computation, or recognizing it but still complying.
Critical strengths
-
The evaluation goes beyond reporting a single accuracy metric. By distinguishing different failure modes such as Hard Fail, CoT Hijacked, and Sycophantic Compliance, the paper provides a more fine-grained view of how models fail, which makes the analysis more comprehensive.
-
The paper contributes a reasonably large benchmark. SaliTrap contains 1,145 examples across four categories of commonsense traps, providing a useful testbed for the community to further study and evaluate this type of reasoning failure.
Critical weaknesses
-
Incomplete coverage of related work.
The related work appears to miss several closely related studies. In particular, prior work has already investigated implicit feasibility constraints and surface-heuristic distraction, false-premise questions, and unreasonable or invalid problem formulations in mathematical reasoning. These works seem directly relevant to the positioning of SaliTrap and should be discussed to better clarify its novelty.- Li, Y., Zhang, L., Jiang, T., Krishnan, R., and Padman, R. The Model Says Walk: How Surface Heuristics Override Implicit Constraints in LLM Reasoning. 2026.
- Tang, Y. BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models. 2026.
- Hu, S., Luo, Y., Wang, H., Cheng, X., Liu, Z., and Sun, M. Won’t Get Fooled Again: Answering Questions with False Premises. ACL 2023.
- Ma, J., Dai, D., Yuan, Z., et al. Large Language Models Struggle with Unreasonability in Math Problems. AAAI 2026.
- Liang, H., Qiang, M., Li, Y., et al. MathDebugger: A Benchmark for Type-Aware Error Detection in Synthetic Math Data.
-
The reliability of the LLM-as-a-Judge evaluation is insufficiently validated.
The five-way behavioral categorization (e.g., Strict Pass, CoT Hijacked, and Sycophantic Compliance) involves fairly subjective judgments, yet the paper does not report any human validation of the Judge labels or human–LLM agreement. Since many of the paper’s main findings rely on these fine-grained categories, it would be important to manually annotate a representative subset of model responses and report agreement with the LLM Judge to verify that the labels are reliable. -
The mitigation experiments are limited to inference-time prompting and do not evaluate capability preservation on normal tasks.
The paper shows that prompting can substantially improve performance on SaliTrap, but does not investigate whether the failure mode can be mitigated through training. More importantly, an intervention that encourages models to constantly question task premises may introduce over-skepticism or unnecessary refusals on otherwise valid tasks. It would be useful to evaluate training-based mitigation together with matched feasible control tasks, to test whether salience bias can be reduced without degrading normal task-solving ability.
Questions for authors
Could the authors provide several complete examples in the appendix, including the original prompt, the model response, and the corresponding Judge label for each of the five behavioral categories? This would help readers better understand how the scoring rubric is applied in practice and assess whether the distinctions between categories align with common human judgment or may be overly strict in some borderline cases.
3 comments
Thank you for the response, the human agreement check, and the examples in Appendix I.
Novelty. I appreciate the open-ended task format and the distinction between missing a constraint and recognizing it but proceeding anyway. Li et al. already showed that a minimal hint can help models apply an overlooked feasibility constraint. I still see substantial overlap in the central finding and view SaliTrap’s main contribution as extending the evaluation to open-ended tasks.
SFT. Table 9 reinforces my concern about over-skepticism. LoRA SFT raised trap avoidance from 1.25% to 70.00%, while correct completion on matched feasible tasks fell from 73.91% to 34.78% and false refusals rose from 0% to 32.61%. I find that trade-off too severe for the reported SFT setup to be a satisfactory mitigation.
For these reasons, I stand by my original rating of 2.
Clarifying SaliTrap’s Novelty and Mitigation Results
Thank you for the follow-up.
Novelty. Li et al.’s paper makes an important contribution through its paired evaluation of hidden feasibility constraints. SaliTrap differs in two respects.
- Research focus. Li et al. show that salient cues can steer models toward actions ruled out by hidden constraints. SaliTrap documents a failure that can occur even after a model explicitly identifies the constraint. The model states that a prerequisite is unmet and still proceeds with the requested calculation or plan.
- Task design. Li et al. primarily evaluate how models answer everyday questions about which action to take. SaliTrap focuses on reasoning tasks that require calculations or multistep planning. This lets us assess whether a constraint the model explicitly recognizes guides its subsequent reasoning.
Mitigation and over-skepticism. We have not claimed that SFT can reliably mitigate salience bias. We added the experiment in Table 9 at your suggestion to test whether LoRA SFT could improve trap avoidance while preserving performance on matched feasible tasks. We agree with your assessment of the trade-off.
Our positive mitigation result concerns inference-time prompting. P1 improved trap avoidance on GLM-5.1 and DeepSeek-V4-Pro. On tinyMMLU (Appendix G.7), GLM-5.1 scored 93% under both conditions, and DeepSeek-V4-Pro’s accuracy decreased by 3 percentage points, from 88% to 85%. P1 thus largely preserved general QA accuracy in the two models tested.
If you have any further questions, we would be happy to discuss them.
Response to Reviewer 448t
Thank you for the careful review and for suggesting ways to strengthen the related work, validate the Judge, and evaluate mitigation.
W1. Related work and novelty
Thank you for pointing out these closely related studies. We have expanded Section 2.2 to clarify how SaliTrap relates to prior work. In The Model Says Walk, Li et al. use an insightful paired design to test whether models change their decisions when an implicit feasibility constraint is removed. SaliTrap studies salience bias in open-ended task execution. Across four types of commonsense traps, we examine how salient numerical or procedural details can draw models into calculation or planning when a necessary prerequisite is unmet. We distinguish failures to recognize the unmet prerequisite from cases in which the model recognizes it and still proceeds. We have also added citations to BrainBench, FalseQA, Unreasonable Math Problems (UMP), and MathDebugger in Section 2.2.
W2. Human validation of the LLM Judge
Appendix F.3 describes the five-way evaluation protocol. We report a human agreement check here. Three annotators with a computer science background each labeled 80 model responses. Across the 240 comparisons between an individual human label and the LLM Judge’s label, 212 matched, yielding 88.3% raw agreement.
W3. Mitigation and feasible-task performance
Thank you for raising the concern about over-skepticism. Appendix G.7 reports our evaluation of P1 on general QA. On tinyMMLU, P1 left GLM-5.1’s accuracy unchanged (93%) and reduced DeepSeek-V4-Pro’s accuracy by 3 percentage points (from 88% to 85%).
We also fine-tuned Qwen3-8B with LoRA SFT on 140 trap–feasible pairs and evaluated it before and after training on held-out seeds under Control.
We report the setup and results in Appendix J.
Question for authors
Appendix I includes one example for each of the five behavioral categories. Each example gives the task prompt, reference explanation, model’s final answer, and Judge label and rationale. The Sycophantic Compliance example also summarizes the relevant part of the model’s reasoning.