acceptodds
Withdrawn from ICLR 2027

Knowing but Not Correcting: Routine Task Requests Suppress Factual Correction in LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

LLMs can correct a false premise in isolation yet comply when the same premise is embedded in a routine task. We call this failure correction suppression. We introduce an automated pipeline combining false-premise generation and task-context sampling, with manual verification, to construct paired isolated and contextualized queries. A benchmark of 300 premises spanning seven error categories and 21 domains reveals suppression across eight models, with rates of 19%–90%. Hidden-state and attention analyses on Qwen and LLaMA support an account of knowing but not correcting: task context diverts attention from the false premise as the response trajectory shifts toward compliance in the middle layers. We propose two training-free interventions. Correction Direction Steering (CDS) injects a correction direction at middle layers. Dynamic Payload Amplification (DPA) uses attention changes to locate payload tokens and amplify their representations at the final layer, eliciting correction without injecting a learned semantic direction. Against ITI, DoLa, and TAE on model-specific suppression sets, CDS achieves the highest correction rates across Qwen, LLaMA, and Gemma, with model-dependent capability trade-offs. DPA outperforms these baselines on LLaMA and Gemma and improves MMLU-Pro accuracy over the unmodified model on both models evaluated for general capability. Both methods incur low inference overhead and produce no observed over-correction in the true-premise evaluation.

Primary area:
interpretability and explainable AI
Keywords:
correction suppression, false premises, factual reliability, sycophancy, activation steering, inference-time intervention, mechanistic interpretability, LLM evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1zNpOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work introduces a benchmark to analyse how LLMs perform premise correction under isolated and contextualised settings. The work finds LLMs usually can correct the false premises in isolated condition but not fail under contextualised condition, which is termed as correction suppression. They further conduct attention analysis in LLMs and claim that the contexts will divert the models' attention to the false premises so that the suppression is incurred. Finally, the work proposes two methods to mitigate the issue, and experiments show the effectiveness.

Critical strengths
  1. This work introduces a meaningful "correction suppression" issue and constructs a benchmark to validate the existence of the issue. This may be helpful for LLM capacity improvement, such as hallucination reduction.

  2. This work supplies an operationable pipeline to evaluate, analyse, and mitigate. This may provide inspiration for further studies on this issue.

  3. Targeted to the issue, the work proposes two methods to mitigate it and experiments show its effectiveness.

Critical weaknesses
  1. The reliable benchmark is the key basis to evaluate the issue, but the construction of the benchmark is not sufficiently introduced. There is much information that should be introduced in detail rather than simply listed out, such as raw materials collection, labelling processing, and verification processing.

  2. The evaluation results are also not sufficient, and some experimental settings may be confused. For example, the models used in Tables 1 and 2 are not consistent, and I think the consistent models' employment will improve the reliability.

  3. The analyses are also confused. For example, Figure 2 compares the hidden states of isolated and contextualised samples, but intuitively, it is worthy to compare between the suppressed and unsuppressed samples under contextualised setting.

  4. The work claims the DPA method follows directly from their attention analysis. According to the experimental results, I find the performance of DPA does not significantly outperform the existing steering methods, which seems to suggest that the analysis does not motivate the underlying reasons for the issue.

  5. To ensure reliability, it is worthy to use varying scaled models.

Questions for authors
  1. How are the false premises and contexts generated, and how do you ensure the contexts align with the premises?

  2. Why do you use the different models, such as in tables 1 and 2? What else meaningful insights evaluation results supply?

  3. How the states and attention of suppressed and unsuppressed samples perform under contextualised setting? Are there any meaningful insights?

  4. What the analysis inspires and why the directly followed DPA performs limited.

  5. The over-correction results should be expanded, with at least more models to be added.

Reviewer_UDWbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper discusses correction suppression, which LLMs identify and rectify false factual premises in isolation, but comply with them when embedded in specific task contexts. The authors build a benchmark and observe suppression across eight models. Based on internal state and attention analyses on Qwen and LLaMA, the paper proposes two methods: Correction Direction Steering (CDS), which adds a steering vector at middle layers, and Dynamic Payload Amplification (DPA), which detects payload positions via attention jumps and amplifies them at the final layer.

Critical strengths
  1. This paper investigates the issue of fact correction in the context of specific tasks and in isolated scenarios, and identifies this called failure correction suppression.
  2. Experiments demonstrate that this phenomenon is common across models of various scales.
  3. The paper proposes two training-free inference intervention methods: Correction Direction Steering (CDS) and Dynamic Payload Amplification (DPA) to recorrect.
Critical weaknesses
  1. Sections 4.1 and Figure 5a show that output crystallizes into compliance in middle layers (L11–L23) and late steering fails (<20% CR at L31). So, that is why the DPA methods also intervened at the 31st level.
  2. Intuitively, this adaptive inference-time enhancement method (DPA) should be more accurate than a global vector estimated from a fixed set of offline samples. Why, then, does DPA actually underperform? The specific reasons require further analysis and explanation.
  3. In Table 4, the effectiveness of CDS cannot be explained by the results from the MMLU-Pro benchmark. Qwen’s performance is lower than all other baselines, while Gemma’s is unusually high. This remains unexplained.
  4. The paper identifies Layer 11 as the optimal intervention layer based solely on an ablation on Qwen. However, since the paper does not provide layer ablation results for other models, it's unclear whether the specific choice of layer 11 in the CDS method can be directly applied to different model architectures.
Questions for authors

see weakness

Reviewer_irVvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates correction suppression, a phenomenon in which LLMs correct false premises in isolation but fail to do so when the same premises are embedded in routine task requests. The authors construct a paired benchmark, evaluate eight LLMs, and analyze hidden-state and attention patterns to investigate the phenomenon. Based on these observations, they propose two training-free interventions, Correction Direction Steering (CDS) and Dynamic Payload Amplification (DPA), and evaluate their effectiveness against existing methods in restoring factual correction.

Critical strengths

1.LLMs may correctly identify false premises in isolation but fail to correct them when embedded in routine task requests. This observation highlights a potential limitation in factual reliability during real-world interactions. The problem is worth investigating and has the potential to support a stronger contribution.

2.The paper constructs a paired benchmark, evaluates eight LLMs, includes additional validation using human-written contexts and documented misconceptions, and investigates internal representations and attention patterns.

Critical weaknesses

1.The paper argues that correction suppression occurs because task context reduces attention to false premises in early layers and shifts the model toward compliance in middle layers. However, the current analyses mainly show correlations between internal representations and correction behavior, rather than establishing that these changes cause correction suppression. More importantly, it is unclear whether CDS and DPA specifically address these proposed mechanisms. For example, CDS constructs a correction direction from isolated and contextualized queries, which differ not only in correction behavior but also in input context. Therefore, the resulting direction may capture contextual differences rather than correction behavior itself. Similarly, DPA improves correction by amplifying selected tokens, but it is unclear whether this improvement comes from addressing the proposed attention dilution or simply from making certain input content more prominent. More targeted control experiments are needed to establish both the causes of correction suppression and why the proposed interventions are effective.

2.The paper presents correction suppression as a new failure mode, but its relationship to existing research on false-premise correction and sycophancy is not sufficiently clear. Previous studies have already shown that LLMs may possess the knowledge needed to identify incorrect information but still fail to correct it, or may accept incorrect claims to comply with users. The main difference in this paper appears to be that the false premise is embedded in a routine task request. However, the authors do not clearly explain why this difference constitutes a distinct research problem rather than another setting of previously studied behaviors. A more direct comparison with existing problem formulations and findings would help clarify what is genuinely new about correction suppression.

3.The paper proposes two methods to recover suppressed correction, but it is unclear whether simpler prompting strategies could achieve similar results. Although the authors include an Instruction baseline, they do not evaluate more structured approaches. For example, the model could first be explicitly asked to check whether the user's request contains incorrect factual claims and then complete the original task based on that verification. Alternatively, factual verification and task completion could be separated into two sequential LLM calls. These methods are straightforward to implement and do not require modifying the model's internal representations. Comparing CDS and DPA with such approaches would help demonstrate whether the proposed interventions offer meaningful advantages in correction performance, general capability, or computational cost.

4.The paper assumes that when a user provides a false premise, the model should correct it. However, whether a factual claim is incorrect and whether the model should actively correct it are two different questions. Not correcting the false premise does not necessarily indicate a model failure. Although the paper evaluates over-correction on true premises, it does not examine situations where the premise is false but correction is not required by the task. The authors should clarify under what task conditions correction is expected and how these conditions relate to their definition of correction suppression.

5.Constructing a new benchmark of 300 false premises across seven error categories and 21 domains is a useful contribution, but it also places higher requirements on the quality and validity of the dataset. The authors have made efforts to ensure data quality through manual verification, deduplication, human-written contexts, and additional evaluations. However, the paper does not sufficiently analyze the difficulty of the constructed questions or how suppression rates vary across different error categories and domains. In addition, some task contexts explicitly include instructions such as "No disclaimers needed" or "Don't explain the basics to me." These instructions may directly discourage correction rather than simply provide routine task context. Although the paper includes context-component ablations, it remains unclear how much of the observed suppression is caused by ordinary task framing versus instructions that discourage additional explanations. Further analysis is needed to establish that the benchmark reliably measures the intended phenomenon.

Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.