- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors investigate how corrupted task specifications can mislead LLMs. The method specifically adds corrupted task specifications or misleading hints to the task prompt in order to mislead and fool the LLM. In their evaluations, they show that LLMs are not able to reliably deal with these scenarios and often falter. To improve the robustness of LLMs, they propose UNHACK, where they specifically add corrupted and misaligned task specifications into RLVR training runs, only rewarding the initial original objective. In doing so, they effectively reduce the compliance with the corrupted specifications, while preserving model capabilities.
Critical strengths
The overall motivation sounds reasonable and important to me, and it is concerning to me that LLMs can be fooled that easily.
Critical weaknesses
I am concerned about the framing of hacking and cheating in the paper. It is unclear to me whether the LLM here is actually trying to hack or cheat the benchmark. If the LLM follows the corrupted or misaligned task specifications, this seems to me more a confusion or a misinterpretation of the LLM.
The paper misses a comparison to reasoning models. I would expect current reasoning models, such as the qwen3.6/3.8 models, to already cope quite well with these scenarios. Is this only an artefact of LLM confusion when they do not have the possibility to think it through?
Questions for authors
What would be the expected optimal model response in these scenarios? Especially if the user input is not coherent, what are the LLMs supposed to do? E.g. if the provided test does not match the task itself. Should the model just ignore the test case or the task instructions? Should it ask for clarification or just proceed with solving? How are questions regarding clarifications treated in your evaluation? How often do LLMs ask for clarification or point out inconsistencies in the provided user input?
Could the authors reiterate the three subsets and clarify what, specifically, each subset tests for with respect to hacking/cheating or misleading models?
Table 1-3 seems rather incomprehensible, and I believe restructuring or visualisation would help the presentation.
Thank you for suggesting a clearer presentation. Your feedback helped us reorganize the main Results section around Pareto plots of task performance and copying or following. The numerical comparisons are collected in our general comment, Additional baseline results (ID), and in the revised version of the paper.
1. Terminology: what we mean by "cheating"
We agree that "cheating" can sound like a claim about intent. This is why we preferred to talk about “specification anchoring”; when we use “cheating,” we follow the same vocabulary of ImpossibleBench [1] and mean the observable behavior, not intent. Regarding our method's name, UNHACK: this stems from the fact that training RLVR with hints or other privileged information can create reward-hacking behavior, and we provide a way around this.
2. Does reasoning resolve this behavior?
This behaviour also occurs in reasoning models: ImpossibleBench reports 33% cheating for o3 even under its strictest instruction to stop when tests conflict with the specification. We use that same instruction as our defense prompt.
3. What should the model do when specifications conflict?
Great question! Standard RLVR leaves this situation unspecified: models are not explicitly trained to handle conflicting guidance, and typically never encounter it during training. We think a model should stick to the task, flag the conflict, and ask for clarification when needed. UNHACK is a first step—it trains models to finish the task without following corrupted guidance. A natural next step would be to reward clarification across turns. Even our strict defense prompt, which tells models to identify inconsistencies, leaves substantial anchoring.
4. How are clarification responses treated?
We don't count a response without a final answer (no \boxed{} field) as copying, but it also gets no credit for solving the problem.
5. How often do models flag the conflict?
As a rough proxy for clarification or confusion, we count responses that mention both the correct value and wrong hint without choosing either. This happens in 1.25% of RLVR responses, 0.96% for UNHACK, and 5.78% for Base. Responses matching this proxy are therefore uncommon. Even across five attempts, a correct answer appears for only 9.7–23.8% of problems under wrong hints, despite much higher accuracy without hints. Together with the low rate of responses expressing uncertainty, this suggests that models tend to follow the wrong hint rather than remain undecided between the two answers.
6. What does each subset test?
Your suggestion prompted us to simplify the presentation by combining the three types of accuracy into one score: their geometric mean, GM3. Each matters for a different reason: accuracy without guidance shows what the model can solve on its own; accuracy with correct guidance shows whether it benefits from useful information; and accuracy with wrong guidance shows whether it resists misleading information. They need to be interpreted together, because high correct-guidance accuracy alone can simply reflect copying. For coding and ALFWorld, the last score counts only tasks completed without following the mistake.
We can then plot GM3 against copying or following in a Pareto plot, making the balance between task performance and resistance to corrupted guidance much easier to see. The main-paper plots also include the additional baselines. Thank you for this suggestion, it helped us make the results much easier to compare.
[1] Ziqian Zhong, Aditi Raghunathan, and Nicholas Carlini. ImpossibleBench: Measuring LLMs’ propensity of exploiting test cases. arXiv preprint arXiv:2510.20270, 2025.