acceptodds
Under review as a conference paper at ICLR 2027

When Context Misleads: In-context Learning with Jurisdiction in Large Language Models

Abstract

In-Context Learning (ICL) has become a cornerstone of modern LLM deployment. However, standard ICL post-training methods have a critical blind spot: they excel at extracting patterns from demonstrations while often neglecting context authority—the ability to determine whether contextual information should govern the final answer. To benchmark this capability, we introduce FakeContext-bench, which contains pseudoscientific claims across seven domains. Our evaluation of commercial and open-source models shows that large-scale pre-training alone is insufficient for reliable context-authority discrimination. Moreover, prevalent ICL fine-tuning methods can increase susceptibility to misleading context, reducing reality accuracy by up to 14.95 percentage points relative to the base model. To address this trade-off, we propose Jurisdiction In-Context Learning (J-ICL), a post-training framework that incorporates context validation into the training objective. Across four model backbones, J-ICL improves ICLEval by an average of 5.84 percentage points and reality accuracy by 9.20 points over the corresponding base models. It also raises the Reality rate by an average of 18.09 points relative to MetaICL and Symbol Tuning. These results show that ICL capability and resistance to deceptive context can be improved together.

Primary area:
foundation or frontier models, including LLMs
Keywords:
In-Context Learning, Misleading Context, Large Language Model Robustness, Knowledge Conflict, Post-Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SDd6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates context-authority discrimination in LLMs. The authors observe that conventional ICL fine-tuning can increase models’ tendency to follow misleading contextual rules. It introduces FakeContext-Bench, containing 3,500 instances across seven domains. Meanwhile, it also proposes Jurisdiction In-Context Learning (J-ICL), a post-training framework that jointly optimizes rule induction and contextual applicability judgment. The authors evaluate J-ICL across four model settings using ICLEval and FakeContext-Bench.

Critical strengths

[S1] The paper highlights an important issue: improving ICL ability can make models more susceptible to misleading demonstrations.

[S2] The paper introduces FakeContext-Bench, a benchmark for evaluating whether LLMs can resist coherent but misleading in-context demonstrations.

Critical weaknesses

[W1] J-ICL primarily applies standard SFT to the Jurisdiction-Contrastive Data, offering limited algorithmic novelty.

[W2] The baseline comparison includes only MetaICL (2022) and Symbol Tuning (2023).

Questions for authors

[Q1] The paper should clarify that its primary contribution is a benchmark rather than the training methodology. J-ICL largely relies on standard SFT with positive and negative pairs.

[Q2] The baseline comparison includes only MetaICL (2022) and Symbol Tuning (2023). Please include more recent and stronger baselines, such as methods for demonstration selection and reranking.

[Q3] The decomposition of ICL into task recognition and task learning [1] could help clarify the proposed task. Please discuss how context-authority discrimination relates to these two components.

[Q4] The paper contains some errors in its mathematical notation, such as the indicator function in Equation (3).

[Q5] Can the proposed approach be applied when model fine-tuning is unavailable, such as when using proprietary models through APIs?

[Q6] The text and number in the figures are too small.

[Q7] If the authors do not release their data, it will be difficult to verify and reproduce the reported results.

[1] What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learning.

4 comments

Thank you for recognizing the importance of misleading demonstrations and the value of FakeContext-Bench. We respond to each question below.

Q1. Contribution and algorithmic novelty.

Our main contribution is to systematically benchmark an underexplored failure mode in ICL: applying coherent contextual rules beyond their legitimate scope. J-ICL complements this benchmark with a simple, targeted data-construction and sampling recipe that mitigates this failure while improving conventional ICL performance. We will clarify this emphasis and better articulate the contributions of both the benchmark and the training approach.

Q2. More recent selection and reranking baselines.

We have added within-case adaptations of ByCS (Bayesian Example Selection Improves In-Context Learning for Speech, Text and Visual Modalities, EMNLP 2024) and ConE (Revisiting Demonstration Selection Strategies in In-Context Learning, ACL 2024). Both use the same unadapted Qwen3-8B model and select only from each case's supplied demonstrations, rather than an external retrieval pool.

For select & rerank, we select two demonstrations on FakeContext-Bench, retaining the first two for cases without explicit demonstration answers. On ICLEval, we retain k = ⌈3N/4⌉ of the N demonstrations and leave copying tasks unchanged. Rerank only preserves all demonstrations and changes only their order. Generation settings and scoring remain fixed within each benchmark.

MethodReality ↑Context ↓Reality Rate ↑ICLEval accuracy ↑
Qwen3-8B Base23.06%60.74%27.51%69.66%
J-ICL28.86%56.14%33.95%76.03%
ByCS (select & rerank)26.23%59.17%30.71%63.87%
ConE (select & rerank)27.57%57.77%32.31%64.26%
ByCS (rerank only)24.26%61.06%28.43%65.69%
ConE (rerank only)26.69%59.60%30.93%65.98%

Under our within-case adaptations, both ByCS and ConE improve Reality Rate over the reported Base result, but neither yields higher ICLEval accuracy, even when reranking retains all demonstrations. One possible explanation is the restricted candidate pool: these methods can only select or reorder the demonstrations already supplied in each case, rather than retrieve more suitable examples from a large external pool. J-ICL tuning and demonstration selection/reranking are also compatible: the former trains the model to assess contextual applicability, while the latter organizes the context presented at inference time. Their combination is a promising direction for future work.

Q3. Relation to task recognition and task learning.

Thank you for pointing us to What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning. We will discuss this distinction in Related Work. Context-authority discrimination concerns whether a recognized or newly learned rule applies to the current query. It complements both components: a model can identify or learn the demonstrated mapping correctly yet apply it outside its scope.

Q4. Mathematical notation.

We will correct the indicator notation in Equation (3) and check all equations for consistency.

Q5. API-only models.

J-ICL requires fine-tuning and cannot be directly applied when an API provides no fine-tuning access. FakeContext-Bench can still evaluate such models through their generated responses. We will clarify this distinction.

Q6. Figure readability.

We will enlarge text, legends, and numerical annotations, adjust crowded layouts, and verify that all figures are readable at normal viewing size.

Q7. Data release.

We will publicly release data used in this work to support verification and reproducibility, and make this commitment explicit in the paper.

The core contributions are the benchmark and the models. Please release them as supplementary material; otherwise, I cannot maintain my score. Also, the baselines you compared are outdated.

Thank you for your helpful suggestions. To facilitate inspection and reproducibility, we have included FakeContext-Bench and its scoring script in the supplementary materials. As the model checkpoints and training data exceed the supplementary submission size limit, we provide them through an anonymous repository, with its link included in the supplementary materials.

For newer baselines, we evaluated within-case adaptations of PDDS, from Performance-Driven Demonstration Selection for In-Context Learning (Wang et al., IJCAI 2026), and DisJEV, from Can Jev Improve Demonstration Retrieval for In-Context Learning? (Song et al., 2026). Both methods select two demonstrations on FakeContext-Bench. On ICLEval, they retain three quarters of the available demonstrations, rounded up to the nearest integer, with copying tasks unchanged.Cluster-based Search, from Rapid Selection and Ordering of In-Context Demonstrations via Prompt Embedding Clustering (Pham et al., ICLR 2025), reorders all demonstrations using next-token entropy.

MethodEvaluationReality ↑Context ↓Reality Rate ↑ICLEval ↑
Qwen3-8B (Base)demos in original order23.06%60.74%27.51%69.66%
J-ICLdemos in original order28.86%56.14%33.95%76.03%
PDDSWithin-case selection and reranking29.00%57.17%33.65%62.70%
DisJEVWithin-case selection and reranking28.29%57.17%33.10%63.58%
Cluster-based SearchWithin-case reranking25.34%60.57%29.50%66.08%

All three baselines improve Reality Rate over Qwen3-8B Base, but their ICLEval accuracy drops. PDDS achieves slightly higher Reality accuracy than J-ICL (29.00% vs. 28.86%), at the cost of a substantial 6.96-percentage-point drop in ICLEval relative to Base (69.66% to 62.70%). J-ICL is the only method in this comparison that improves both Reality Rate and ICLEval over Base.

J-ICL fine-tunes the LLM on downstream data, so it is unfair to compare it with these training-free baselines. Moreover, training the LLM defeats the very purpose of ICL, whose core advantage is being training-free. Besides, as the authors themselves state, the core contribution is only a dataset, and the algorithm directly uses standard SFT, so I see no algorithmic contribution. I therefore keep my score.

Reviewer_6Pw9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies whether large language models can determine when rules inferred from in-context demonstrations should apply to a query. It proposes FakeContext-Bench, comprising 3,500 instances across seven domains. The authors find that conventional ICL tuning can increase susceptibility to misleading demonstrations. To address this issue, they propose Jurisdiction In-Context Learning (J-ICL), which combines applicable and inapplicable contextual-rule episodes through structured sampling and a query-only language-modeling objective.

Critical strengths

The proposed FakeContext-Bench contains 3,500 instances across seven domains, organized into 700 families. Conventional ICL benchmarks typically assume that the rule demonstrated in context applies to the query. FakeContext-Bench presents internally consistent demonstrations of a misleading rule while asking a query governed by a reality-grounded rule.

Critical weaknesses

Several figures are difficult to read. Figures 2(c) and 3(a) remain difficult to read after zoomed.

The contribution of the paired training structure is not clearly isolated. The paired objectives average independent query-answer losses without an explicit cross-example constraint. Moreover, removing a training component and redistributing its weight also changes the effective amount of negative-jurisdiction supervision. The component ablations do not cleanly distinguish the benefit of pairing from that of the training mixture.

The main benchmark consistently rewards reality-grounded answers, while matched positive and negative training episodes share the same canonical target. This leaves open whether the improvement reflects applicability judgments or a stronger preference for canonical rules.

Reality Rate excludes responses matching neither registered answer and cannot independently separate applicability failures from knowledge or reasoning limitations. Human validation covers only 100 mixed responses and shows asymmetric agreement: 47/54 for human-labeled Reality responses versus 43/43 for Context responses. This raises concerns about whether differences in response style influence model comparisons.

Questions for authors

Could you clarify what paired sampling contributes beyond the underlying data mixture? In particular, can the component ablations preserve the effective positive/negative supervision weights and training budget while changing pair correspondence?

Could you evaluate matched cases that keep the demonstrated rule fixed but switch whether it applies to the query, reporting the proportion of pairs answered correctly in both conditions?

4 comments

Thank you for recognizing the benchmark's value and for the specific questions about sampling, applicability, and evaluation.

1. What paired sampling contributes.

Thank you for highlighting the need to separate the effect of pairing from that of the supervision mixture. We agree that component-removal ablations change the mixture and therefore cannot isolate pairing alone. J-ICL uses standard, independently computed query-answer losses without an explicit cross-example constraint; pairing determines which related episodes contribute to the same optimizer update.

We compare two controls on Qwen3-8B. Independent-sampling SFT preserves the overall positive/negative supervision ratio and training budget but samples episodes independently. Shuffled-Pair provides a stricter control: it preserves all sampled examples and shuffles their grouping across optimizer updates, breaking matched correspondence within each update.

MethodICLEval ↑Reality ↑Context ↓OtherReality Rate ↑
Independent-sampling SFT71.94%27.80%57.31%14.89%32.66%
Shuffled-Pair73.92%28.94%56.37%14.69%33.92%
Full J-ICL76.03%28.86%56.14%15.00%33.95%

Compared with independent-sampling SFT, Full J-ICL improves ICLEval by 4.09 percentage points and Reality Rate by 1.29 points. Under the stricter Shuffled-Pair control, matched co-sampling improves ICLEval by 2.11 points, while Reality Rate remains nearly unchanged (+0.03 points). Thus, the clearest contribution of pairing is improved conventional ICL performance while maintaining comparable resistance to misleading context, rather than a substantial additional gain in resistance itself.

We will clarify this distinction in the method section and include both controls in the revised experimental results.

2. Applicability judgment versus a preference for canonical answers.

Our original design uses ICLEval to assess learning from applicable contextual rules and FakeContext-Bench to assess resistance to inapplicable rules. Because FakeContext-Bench focuses on scientific and factual questions, making its misleading rules applicable requires fictional reframing, such as specifying a simulator that implements them. Although useful for controlled testing, this creates a less natural setting that is less representative of the real-world scenarios motivating our benchmark. For this reason, our original evaluation primarily used ICLEval to assess context learning and following, rather than including this reversal experiment.

To directly answer your question, we have added the requested matched evaluation. The demonstrated rule, demonstrations, and task parameters remain fixed, while the statement and query scope change so that Context becomes correct. PairAcc requires both answers in a pair to be correct: Reality in the original condition and Context in the applicable condition.

MethodOriginal: Reality accuracy ↑Applicable: Context accuracy ↑PairAcc ↑
Base23.06%78.06%15.80%
MetaICL21.31%80.17%14.40%
Symbol Tuning8.11%76.66%2.34%
J-ICL28.86%80.51%18.69%

On Qwen3-8B, J-ICL improves PairAcc over Base and MetaICL by 2.89 and 4.29 percentage points, while also achieving the highest Context accuracy when the alternative rule applies. This provides evidence beyond a stronger preference for canonical answers. We will include these results in the revision.

3. Reality Rate and scorer validation.

We agree that Reality Rate, defined as R/(R+C), is a conditional measure and cannot independently separate applicability errors from knowledge or reasoning limitations. We will make this explicit and present it alongside unconditional Reality, Context, and Other rates. The matched evaluation above also reports unconditional accuracies and PairAcc.

To supplement the original 100-response validation, we conducted an additional human review of 200 responses. The scorer agrees with the reviewed labels on 181/200 responses (90.5%), with Cohen's κ = 0.8100. Agreement is 64/69 (92.75%) on human-labeled Reality responses and 117/131 (89.31%) on human-labeled Context responses. We will include these supplementary validation results in the revision.

4. Figure readability.

We will enlarge the fonts, labels, and numerical annotations in Figures 2(c) and 3(a), adjust their layouts, and check readability at normal viewing size throughout the paper.

1. The new ablation study result is promising and proved the full design of J-ICL is reasonable. Please consider include this part in your manuscript.

2. Can you explain more on why you decide to adopt PairAcc as an assessment, considering Symbol Tuning perform quite differently from other methods

3. The additional human review is a good result. But I am also curious about how you designed to choose certain people to carry out the review. Please include more details on this part of design in your revision.

4. Please show the new design of figures in your revision.

Overall, the comment answers most of my questions and concerns. Please update your revision version of pdf manuscript. I will raise score accordingly.

Thank you for reviewing the new results and for your suggestions on the revision. We have uploaded the revised manuscript.

  1. Paired sampling. We included the Shuffled-Pair control in Section 5.2 and Table 5.
  2. Why PairAcc. We use PairAcc to measure how often a model answers both versions of the same case correctly when the applicability of the demonstrated rule changes. The two condition-specific accuracies show where errors occur, while PairAcc shows how often the model gets both right on the same pair. Symbol Tuning illustrates this distinction: it achieves 76.66% Context accuracy when the rule applies and 8.11% Reality accuracy when it does not. It answers both versions correctly on 2.34% of pairs. We report all three metrics in Table 8. This low PairAcc partly reflects Symbol Tuning’s susceptibility to misleading demonstrations in the original benchmark setting, where it follows the demonstrated rule in 74.94% of cases even though the rule does not apply.
  3. Human evaluation. Three co-authors who were closely involved in designing FakeContext-Bench annotated the responses. We selected them for their familiarity with the task setup and scoring definitions.
  4. Figures. We enlarged the legend and improved the contrast of the annotations in Figure 2(c). We also enlarged and bolded the legends in Figure 3(a).

Appreciate for all your effort, I updated my score since the additional result seems convincing.

But I still wish to see if you can expand the study to more model families despite just Qwen and Llama. Also expanding to more parameter size.

Reviewer_UdnJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether models apply coherent contextual rules outside their intended scope. It introduces FAKECONTEXT-BENCH, with 3,500 instances from 700 theory families, and J-ICL, which trains on applicable and inapplicable demonstrations using answer-only supervision and structured sampling. Across four configurations, J-ICL improves ordinary ICL and unconditional Reality accuracy over base models. The main remaining question is whether it also follows alternative rules correctly when they become applicable.

Critical strengths
  1. A useful, coherent evaluation. Registering both reality-grounded and context-induced answers distinguishes following an inapplicable rule from unrelated errors (§3). The benchmark, training intervention and scope controls address the same practical failure mode.
  2. Benefits beyond the base-model comparison. J-ICL improves both evaluated capabilities, and comparisons with MetaICL, Symbol Tuning and same-data independent-sampling SFT support the training recipe's value (Tables 3–4).
Critical weaknesses
  1. Correct applicability-dependent switching remains unverified. Table 6 compares two conditions where Context is correct in both. Its high prediction invariance does not establish correctness or show that a model switches from Reality to Context when the same rule becomes applicable.
  2. The broader contribution needs clearer boundaries. The matched controls support the recipe, but do not establish a capability distinct from instruction or scope following, or its additional benefit on an already instruction-tuned checkpoint. This limits the broader interpretation, not the reported comparisons.
Questions for authors
  1. What are the absolute Context and Other rates in Table 6, and how often do matched original/applicable cases correctly switch from Reality to Context?
  2. Should J-ICL be understood as a scope-sensitive training recipe from base checkpoints, or is there evidence for an additional capability beyond instruction following?
2 comments

Thank you for recognizing the benchmark design and the benefits of J-ICL. We address your two questions below.

1. Correct applicability-dependent switching and the interpretation of Table 6.

Our original design uses ICLEval to assess learning from applicable contextual rules and FakeContext-Bench to assess resistance to inapplicable rules. Because FakeContext-Bench focuses on scientific and factual questions, making its misleading rules applicable requires fictional reframing, such as specifying a simulator that implements them. Although useful for controlled testing, this creates a less natural setting that is less representative of the real-world scenarios motivating our benchmark. For this reason, our original evaluation primarily used ICLEval to assess context learning and following, rather than including this reversal experiment.

To directly answer your question, we now preserve the demonstrations and task parameters while modifying the statement and query scope so that Context becomes correct. We report PairAcc, which requires answering Reality in the original condition and Context in the reversed condition for the same pair.

Results on Qwen3-8B are:

MethodOriginal: Reality accuracy ↑Applicable: Context accuracy ↑PairAcc ↑
Base23.06%78.06%15.80%
MetaICL21.31%80.17%14.40%
Symbol Tuning8.11%76.66%2.34%
J-ICL28.86%80.51%18.69%

J-ICL achieves the highest accuracy in both conditions and improves PairAcc over Base and MetaICL by 2.89 and 4.29 percentage points, respectively. These results provide direct evidence of improved applicability-dependent switching.

Table 6 addresses a separate question: whether predictions depend on official-sounding wording when applicability is held fixed. Context is correct under both the private-applicable and official-applicable conditions. The absolute rates are:

ModelConditionContext ↑Reality ↓Other ↓
BasePrivate-applicable68.37%21.91%9.71%
BaseOfficial-applicable68.23%22.77%9.00%
J-ICLPrivate-applicable69.57%22.97%7.46%
J-ICLOfficial-applicable69.54%22.91%7.54%

Each condition contains 3,500 cases. J-ICL's Context accuracy differs by only 0.03 percentage points, with 96.77% paired prediction invariance. This supports robustness to these wording changes, but does not itself demonstrate correct switching. We will clarify this distinction and include the absolute rates and the new PairAcc results.

2. Scope of the contribution.

Thank you for raising this important question. To examine whether J-ICL offers benefits beyond existing instruction tuning, we additionally train it from the instruction-tuned Qwen3-8B checkpoint. J-ICL improves ICLEval by 5.15 percentage points and Reality Rate by 1.85 points. We report both initialization settings below for comparison.

InitializationMethodICLEval ↑Reality ↑Context ↓OtherReality Rate ↑
BaseOriginal69.66%23.06%60.74%16.20%27.51%
Base+ J-ICL76.03%28.86%56.14%15.00%33.95%
InstructOriginal65.05%24.97%59.77%15.26%29.47%
Instruct+ J-ICL70.20%26.51%58.14%15.34%31.32%

We would also like to clarify the distinction between following instructions and assessing demonstrated rules. In FakeContext-Bench, the user presents an incorrect rule as valid and reinforces it through demonstrations; answering correctly requires rejecting that rule when it does not apply to the query. Our matched evaluation further tests whether the model follows the same rule when it becomes applicable. Together, these results support improved applicability-sensitive behavior, with gains from both base and instruction-tuned checkpoints. We will include these results and clarify the scope of our contribution in the revised paper.

Thanks for the detailed response. I would keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.