acceptodds
Under review as a conference paper at ICLR 2027

When Demonstrations Fail: Representation-Resource Matching in In-Context Adaptation

Abstract

In-context learning lets a language model acquire a new task from a handful of demonstrations, yet some mapping tasks are learned and others of near-identical surface form are not. We ask a causal question that existing work, which describes the representation in-context learning forms, does not: which representation source makes a demonstration usable? Grounded in low-resource multilingual translation, we find that correct source–target correspondence is a graded adaptation signal, and that its benefit is gated by representation accessibility: injectively corrupting the source—holding content, demonstrations, and decoding fixed—collapses it, and this gating is selective: a surface bijection survives the corruption that collapses a semantic mapping. In controlled tasks, a computed intermediate constraint separates via direct-intermediate probes and an explicit rule control; a held-out-query control shows the numeric task’s apparent rule induction is largely inflated by support overlap, while a relational task, whose query pair is held out by construction, reveals a genuine computation failure, and supplying the correct intermediate rescues both. Finally, a cross-model check shows the selective dissociation and the intermediate rescue are positive across five model families (two weaker; Appendix), though their magnitudes do not rank with the tested scale range. Together, these intervention-based results identify a representation-resource constraint on in-context adaptation—an intervention-based behavioral taxonomy of whether the source a task demands is available during adaptation—that scalar accuracy does not capture.

Primary area:
foundation or frontier models, including LLMs
Keywords:
in-context learning, representation learning, large language models, mechanistic interpretability, few-shot learning, multilingual machine translation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_BQ2WOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper sorts in-context tasks by the source their output depends on (surface identity, pretrained semantics, or a computed intermediate) and probes each with interventions on Qwen3-8B, including a correspondence dose in low-resource MT, byte corruption of the source, and held-out-digit and intermediate-rescue conditions on two synthetic tasks, plus a five-family check. Code is described as a supplementary archive; no repository is linked.

Critical strengths
  1. Some confound controls are well chosen. Tokenization-preserving word substitution sends semantic q=8q=8 accuracy to roughly chance (0.18–0.28) in all five families (Table 8), ruling out tokenization as the cause of the corruption collapse.
  2. The held-out-digit control (Table 4) exposes a support-overlap artifact, and the paper retracts its own feature-identification reading accordingly.
Critical weaknesses
  1. Each finding restates something already known. That correct pairings help translation is expected. An unreadable source breaking semantic mappings but not arbitrary symbol bijections mirrors the task-recognition/task-learning split of Pan et al. (2023), and an 8B model answering with thinking disabled failing a multi-step composition that a supplied intermediate fixes matches Dziri et al. (2023) and the scratchpad gains of Nye et al. (2021). The framework adds no predictive content, since Section 3.1 concedes that Am\mathcal{A}_m is read off the same interventions it explains, and Appendix E finds a held-out signal only for the surface route (N=5N=5).

  2. The computed-intermediate claim outlives the paper's own controls. With the query digit held out, numeric joint, explicit-rule, and direct accuracy fall to 0.025/0.205/0.365, yet the five-family rescue (Appendix D) is still measured against the overlapped joint baseline. Appendix K blames support overlap, but K=12K=12 (0.525) gives no gain over K=8K=8 (0.58) though more demonstrations only raise overlap. The relational joint score moves from 0.48 to 0.58 with decode budget and is omitted cross-family for truncation, while enabling thinking lifts numeric joint accuracy from 0.58 to 0.78 (Appendix B). What remains looks like a limit of single-pass decoding, not of availability during adaptation.

Questions for authors
  1. At K=8K=8, what is joint accuracy on episodes where the query digit recurs? With 80% recurrence and 0.025 otherwise, the reported 0.58 implies about 0.72, which reuse alone doesn't explain.
  2. Does the relational direct probe stay near chance with thinking enabled?
  3. What are the absolute byte-corrupted chrF++ scores behind Table 3? A floor would make the Δ08\Delta_{08} collapse uninformative.
Reviewer_ibA2OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Why do some samples for in-context learning support the downstream task, while other seemingly similar samples do not? The authors propose to factorize ICL examples into three functional parts: superficial features, pretrained semantics, and computed intermediates. The authors hypothesize that an underlying task requires particular representations governed by these factors, and that these are made available via ICL conditioned on chosen prompting and decoding policies. The authors examine this using low-resource to Chinese translation under four tasks: token bijection, semantic-category mapping, digit-residue classification, and inference over a hidden order. The show that correct correspondence improves translation, while source corruption reduces its benefit and affects semantic mapping more strongly than surface mapping. Supplying correct intermediates substantially improves the synthetic tasks.

Critical strengths

Section 5.1 distinguishes the effect of correct source–target pairing from demonstration volume and prompt format. In particular, the same-format control retains the paired layout while changing target correspondence and reports broad improvements with correspondence. It is the most robust result and is operational on a real task.

The numeric holdout experiment in Finding 3 identifies an important evaluation shortcut: joint accuracy falls from 0.58 to 0.025 when the query digit is excluded from demonstrations (Table 4). The authors acknowledge this refutes their initial feature-identification interpretation.

Critical weaknesses

The writing is very difficult to follow, and I think the project would benefit from a graphical figure showing what the hypothesis is and what the four experiments are that are being used to assess it. There are multiple experiments, languages, perturbations, etc... and I found myself scrolling around a lot to pull it all together.

The additional contribution over existing inference/verbalization accounts isn't clear in light of the Finding 3 results... Section 2.3 retracts the feature-identification interpretation. The remaining relational results are useful, but the paper does not clearly establish what further diagnostic insights these provide. Section 3.1 also acknowledges that representation availability is defined retrospectively from the observed intervention outcomes.

Questions for authors

After withdrawing the feature-identification interpretation, which experimental findings go beyond the inference/verbalization decomposition of Tao et al. (https://arxiv.org/abs/2410.09349), or have we largely just reproduced those findings here under slightly different names?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.