- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether large language models adjust their decisions to implicit feasibility constraints, exemplified by recognizing that visiting a nearby car wash requires bringing the car. It argues that accuracy on constraint-present questions cannot distinguish constraint-sensitive behavior from a general preference for one option. The main contributions are a paired evaluation framework measuring selectivity, over-activation, and under-activation; the 500-prompt Heuristic Override Benchmark (HOB); and CORE-100, a benchmark of 100 human-validated scenarios with minimally edited constraint-present/constraint-absent pairs and additional explicitness and salience controls. The evaluation comprises log-probability sweeps on seven open-weight models, HOB experiments on fourteen models, and a preregistered CORE-100 evaluation of ten models. The results reveal both failures to apply relevant constraints and continued application after those constraints are removed. Active-only accuracy exceeds paired balanced accuracy by 4.3–11.1 percentage points across all ten CORE-100 models and changes their ranking. Furthermore, only one of twelve preregistered prompting comparisons significantly improves selectivity after multiple-testing correction. A separate judge audit shows that sharing constraint metadata across conditions can bias LLM grading. Together, these experiments provide behavioral evidence that hidden-constraint sensitivity requires paired evaluation.
Critical strengths
A consequential finding about evaluation validity. The paper shows that high accuracy when a hidden constraint applies can coexist with systematic over-application when it is absent. On CORE-100, active-only accuracy exceeds paired balanced accuracy for all ten models and changes their ranking. I consider this a substantive reason for acceptance because it changes how apparent reasoning success and prompting improvements should be interpreted.
Careful experimental controls supporting the central finding. CORE-100 uses minimally edited pairs with human-validated reversals in the preferred action, complemented by explicitness and salience controls. Preregistered main analyses, repeated sampling, and scenario-level uncertainty estimates strengthen the comparisons. These choices increase my confidence that the reported evaluation gap reflects a meaningful behavioral limitation, rather than merely uncontrolled differences between prompts.
Critical weaknesses
The interpretation of the sweep experiments relies on an overly strong gating criterion. Sections 2–3 associate constraint gating with a flat log-odds curve. However, a model can correctly prefer the feasible action at every distance while its confidence varies with distance. Parallel conflict and control curves therefore do not establish that the constraint merely shifts a heuristic preference rather than appropriately governing the choice. This weakens a prominent explanatory conclusion, although it does not invalidate the paired-performance results.
The distinction between feasibility constraints and practical preferences remains insufficiently resolved. Human agreement establishes a preferred action, but does not necessarily establish that the alternative violates a feasibility constraint. For example, replacing a couch with a lamp may change carrying costs and convenience rather than a strict prerequisite. Without distinguishing these cases, some measured “over-activation” could reflect different practical preferences. This matters because it limits how directly benchmark errors can be interpreted as failures of hidden-constraint reasoning.
Questions for authors
the criterion for constraint gating. In Sections 2–3, what justifies requiring a flat log-odds curve for constraint gating, rather than a curve that remains below zero at every distance? A model could consistently choose the feasible action while its confidence varies. This clarification would determine whether I interpret the sweeps as evidence against gating or as evidence of residual cue sensitivity.
what the benchmark’s validation establishes. How does the existing annotation procedure distinguish an infeasible action from a feasible but less-preferred action, particularly in examples such as the couch/lamp pair? Appendix H describes the gold labels as validated everyday preferences. This distinction affects how confidently I can interpret measured over-activation as a failure of constraint reasoning rather than disagreement about practical preferences.
2 comments
Full Results Tables for Reference
To make your second question easy to check, here are the complete stratified results and the per-model breakdown from App. J.3, as reported in the revised PDF.
Over-activation by strength of the human preference and by strictness (App. J.3, Table 25, p. 31). pR: share of trials choosing the constraint-heavy action on Removed items; models: mean over the ten models.
| Stratum | n | Raters pR | Models pR [95% CI] | Models with pA − SBA > 0 |
|---|---|---|---|---|
| All scenarios | 100 | 0.002 | 0.219 [0.17, 0.27] | 10/10 |
| Removed: mean rater confidence ≥ 4.5 | 27 | 0.000 | 0.051 [0.02, 0.10] | 5/10 |
| Removed: mean rater confidence < 4.5 | 73 | 0.003 | 0.282 [0.22, 0.35] | 10/10 |
| Active constraint coded strict prerequisite | 77 | 0.000 | 0.203 [0.15, 0.26] | 10/10 |
| Active constraint coded strong cost | 15 | 0.013 | 0.296 [0.15, 0.46] | 10/10 |
| Coders disagree | 8 | 0.000 | 0.230 [0.07, 0.44] | 10/10 |
The 4.5 confidence threshold was chosen post hoc, so that split is exploratory.
Over-activation per model (Table 26, p. 31):
| Model | All 100 | Strict (77) | Cost (15) |
|---|---|---|---|
| DeepSeek V4 Pro | 0.117 | 0.111 | 0.133 |
| DeepSeek R1 | 0.121 | 0.098 | 0.181 |
| Gemini 3.1 Pro | 0.167 | 0.165 | 0.248 |
| Claude Opus 4.6 | 0.184 | 0.187 | 0.200 |
| Kimi K2.5 | 0.192 | 0.174 | 0.295 |
| Claude Sonnet 4.5 | 0.226 | 0.221 | 0.238 |
| GPT-OSS-120B | 0.237 | 0.198 | 0.371 |
| Qwen3-32B | 0.249 | 0.210 | 0.476 |
| GPT-5.4 | 0.306 | 0.323 | 0.295 |
| Llama 4 Scout | 0.394 | 0.345 | 0.524 |
| Raters (in-sample) | 0.002 | 0.000 | 0.013 |
Thank you for the careful review. Both questions helped us state the claims more exactly.
Q1 / W1. The criterion for constraint gating. We agree that the criterion should be a curve that stays below zero at every distance, and §2 (p. 3) now defines gating that way. A model can choose the feasible action throughout while its confidence varies, and we no longer treat a residual slope as an error.
Read with this criterion, the sweeps show both of the things you distinguish.
The matched control also shows how little the constraint adds on the canonical example: on the car wash, four of seven models show no detectable shift relative to it.
GPT-OSS-120B is the model you have in mind: it chooses Drive at every distance while its margin still moves. §3 (pp. 3–4) counts it as behaviorally correct. It also prefers driving for the gift-card control at 100 m, so its correct answer to the test question does not by itself show that the constraint drives the decision (§3, p. 4).
Q2 / W2. Infeasible actions versus feasible but less-preferred ones. The annotation establishes which action raters prefer in each version. It does not certify that the other action is infeasible, and the paper says that golds are validated preferences (§2, p. 3; §9, p. 9; App. H, p. 27). Over-activation is accordingly defined as a decision that does not update when the constraint-relevant fact is edited. Driving a lamp is feasible; the error is choosing as if the couch were still there, when the action raters prefer has reversed.
Does over-activation then reflect a difference in practical preference? Three results in App. J.3 (p. 30) say no.
Your couch example was labeled a strict prerequisite by both coders, and all five raters chose "walk" for the lamp version (strictness labels and rating sheets in the supplementary material).
Thank you for raising both points; the paper is more precise for them.