acceptodds
Under review as a conference paper at ICLR 2027

Informativity Thresholds for In-Context Learning

Abstract

When does in-context learning (ICL) succeed, and when does it fail? We give a quantitative answer by building an informativity theory of ICL on Willems' fundamental lemma: the prompt is recorded input-output data, the lemma specifies how much excitation that data must contain before the continuation is determined, and the conditioning of the data sets how error grows as information thins. Measured on this scale, a transformer trained on the task class nearly matches the analytic Bayes predictor and beats the model-free data-enabled predictor by 3-11x where only identification is possible, collapsing near the point where the prompt first determines the answer. The theory yields one universal law (a prompt that leaves the answer undetermined within the task class defeats every architecture) and one model-relative law: identification provably needs less context information than model-free interpolation, so how much is enough depends on the predictor's prior. Pretrained open-weight LLMs obey the same two laws in two regimes. On latent-system identification probes (2.7B-120B, five families) their error is flat in the prompt's information content, 11-18x above least squares regardless of scale or instruction tuning and 47-75x with a 3072-token reasoning budget: the bottleneck is the model's extraction, not the prompt. On short-memory sources, which their prior covers, error falls as context information grows and, where emissions are ambiguous, ends below both fixed-order n-gram baselines (four models, 2.7B-14B). Swapping only the training prior on one fixed transformer moves the same task between the regimes. Prompt engineering can supply information; whether a model can use it is fixed at training time.

Primary area:
foundation or frontier models, including LLMs
Keywords:
in-context learning, informativity, persistency of excitation, Willems' fundamental lemma, data-enabled predictive control, large language models, hidden Markov models, Bayes-optimal prediction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_q3JDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper claims to introduce a theory by which to characterise ICL's limits.

Critical strengths

The paper is grammatically well written, as well as well-motivated. Exploring when ICL breaks is decidedly an interesting problem.

Critical weaknesses

I find a few issues with this paper. First, the writing is extremely unscientific and subjective, and technically I would argue that the results/experiments (which are underspecified) do not support the thesis. But, more importantly, the claims are very strong for what the paper shows (or it is already known).

I do appreciate the effort the authors put into this specific problem, which--at least to me--is compelling. However, as it stands, the authors need to put in more effort in this work so it meets the bar. Here's how I would improve this paper:

Regarding writing:

  1. The paper is severely informal for an article showcasing theory.
    • For example, Eq. 1. There is a statement that n∈Nn \in \mathbb{N}, but then the rest of the equation is entirely underspecified (is AA a vector? A complex matrix? A tensor over Q\mathbb{Q}? (L127). On the same line, ℓ(S)\ell(S) is a function mapping what to what? What is the observability index? These are just examples from the first paragraph.
    • The proof style is not formal. There are two propositions (without proofs or justifications) and then a theorem (unrelated) without even a proof sketch. Full proofs do belong in the appendix, but you need at least a sketch. The propositions are stated to be the 'ingredients' for the treatment,
    • You need to introduce your background better. The introduction starts throwing in terminology without providing an intuitive (or motivational) background first. The conclusion is essentially symbolic in this work. It is meant to tell us what the outcomes of your work are, not closing a narrative (which is as it reads now).
  2. The writing itself is unscientific:
    • The section titles are leading. You should put the interpretation of the results in the discussion section, not in the results headers. For example,L350: 'Head to head with Dai (2025): the boundary is latent memory depth'
    • There are quite a few references missing with respect to how in-context learning has been characterised. I would argue that most of the works (including the ones missing) that have done that also include the limitations for ICL.
    • Do not start an abstract with a question.It is not just style (which is unacademic), it simply does not fulfill the definition of an abstract: a condensed version of your paper. Clip out numbers and keep it readable.
    • Claims are too strong or obvious:
  3. Additionally:
    • It is unclear to me what are your experiments. You have the models (without citations), and then a slew of numbers without actually telling the reader what you measured, how you measured, which hypothesis you are testing, etc.

Fundamentally, I think that this paper lacks positioning, and it is not objective enough to be a scientific contribution. The problem is interesting but it really seems that it needed considerable more time to be a paper at the level of ICLR.

Questions for authors

I think one of the most fundamental questions I have for the authors is how does this compare to previous work. I see some minor comparisons in the related work section, but it does not fully position it with respect to other existing paradigms/explorations--what are they lacking? What are your shortcomings?

Reviewer_Wau2OpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Very nice paper asking and, in my opinion, providing an interesting answer to a key question in ICL studies: how does ICL emergence depend on the data? What properties of the data are important and how do those properties affect what is learned.

Critical strengths

I really like this paper. This is very interesting and a very nice work answering an important question. I think the paper is very systematic in its approach and does not overclaim (too much) what it does. All related works that I expected to be present were present. All connections to classic and well known statistical estimators were mentioned and discussed as well, much to my joy.

Critical weaknesses

The main weakness of this paper is that I think the relevance and interest of its topic and claims are obfuscated by a difficult-to-parse empirics section and empirical results, particularly the content of page 7. There are many different things going on here and it would be wise to spend more space expanding this.

In terms of actual complaints, I will say I'm not convinced by the k_id value presented in Figure 1 being the actual location of the collapse. Qualitatively of course I agree there is a "knee" here but I think k=2-3 is not supported by this plot itself. I think this could benefit from stronger numerical methods or a systematic way of selecting where the actual knee is from data, and then comparing that numerical location to the predicted value.

Questions for authors

No further questions other than rewrites and slightly-strange Figure 1 mentioned above.

Reviewer_aALSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a fundamental question: how much information does a large model need in order to learn a rule from examples provided in context?

Drawing on system identification and information sufficiency theory, the authors divide prediction methods into two categories: those that first infer the underlying rule from the context and then predict based on it, and those that do not explicitly infer the rule but instead combine or interpolate from existing data. The authors prove that these two specific types of methods have different data requirements: the former depends primarily on the intrinsic complexity of the system itself, while the latter requires more information as the prediction horizon grows. They then validate these theoretical results using specially trained Transformers, hidden Markov models (HMMs), and pretrained large language models, and examine how the training data distribution influences the model's prediction behavior. The core findings have some value, and the experimental results are reasonably convincing. However, evidence remains insufficient regarding what mechanism large models actually employ internally, as well as regarding the theoretical conclusions on discrete sequences; moreover, the conditions and model categories tested are not broad enough to generalize to all models.

Critical strengths
  1. The research angle is genuinely valuable.

The paper makes a useful distinction between two questions: whether the context itself provides sufficient information, and whether the model has the ability to discover the rule from that information. Compared to simply increasing the number of in-context examples, the authors study this topic by controlling the richness of the input signal, which allows for a more precise analysis of what actually limits the model.

  1. The linear system experiments are well designed.

The authors test models on entirely new inputs, vary the prediction length, and compare different prediction methods. Together, these experiments demonstrate that a specially trained Transformer can indeed infer some system rules from context, rather than merely combining existing data. In particular, Figures 1, 4, and 15 clearly show where the model's prediction ability improves markedly, and this aligns closely with the theoretical system identification threshold.

  1. The experimental controls are adequate, and the results are reported transparently.

The authors test not only classical algorithms but also whether the models approximate a Bayes-optimal predictor, and they vary the training data distribution to observe changes in model behavior. They also proactively report results that did not meet expectations, including some experiments that failed the preregistered criteria and the limited explanatory power of the condition number for model error.

Critical weaknesses
  1. The proof of the key theoretical result for discrete sequences is incomplete.

Theorem 3 (page 14 of the PDF) states that for certain discrete sequence models, high-accuracy prediction is achievable with relatively little data. However, the cited existing theory actually requires additional conditions: for example, rank requirements on certain matrices, probability positivity conditions, and other parameters that affect sample complexity. The paper does not adequately establish whether these conditions are satisfied.

For instance, in the experiments there are cases with system rank 4 but only 3 symbol categories (page 6, lines 316–317). This cannot directly satisfy the full column-rank requirement on the single-symbol emission matrix assumed by the cited HMM theory. The authors might address this through more general WFA theory or blocked observations, but a clear argument is needed. Furthermore, there are missing proof steps between estimating statistics from a single sequence and guaranteeing that the final predicted probabilities are sufficiently accurate.

Corollary 2 (same page) has a similar problem. The claim that certain simple statistical methods may require exponentially more data is currently argued mainly by counting the number of possible history sequences, without providing rigorous hard cases or lower bounds on the error.

In summary, I believe there is a substantial gap in the proofs. This does not mean the conclusions are necessarily wrong, but the current proofs are insufficient to support such strong claims.

  1. The paper draws too sharp a distinction between "interpolation" and "system identification."

What the paper proves is a difference between two specific methods (system identification and a specific predictor). However, in the discussion the authors generalize this result to two broader mechanisms, "interpolation" and "identification." The problem is that these two categories do not necessarily have a clear boundary. For example, least-squares regression can be understood either as estimating model parameters or written in the mathematical form of kernel regression. Therefore, a method having a kernel-regression form does not imply it has not learned the underlying rule. Likewise, the observation that pretrained LLMs' predictions are closer to nearest-neighbor or kernel-regression algorithms, or that their hidden features more readily decode into the predictions of such algorithms, does not directly prove that the model internally operates according to these algorithms. The model might have performed only partial system identification, or combined several prediction strategies.

The paper should clearly distinguish among:

  • the proven difference between the two specific prediction methods;
  • the model behaviors observed in experiments;
  • the authors' further speculation about the model's internal computational mechanism.

The first two points are well supported; the third currently lacks decisive evidence.

  1. The interpretations regarding training data, memory depth, and condition number go beyond what the experiments can establish.

Table 1 shows that, with the model architecture and training pipeline held fixed, changing the training data distribution does change the model's prediction behavior. However, this only demonstrates that the training data distribution matters; it does not prove that the mechanism a model adopts is fully determined by the training data. Model capacity, architecture, and optimization may also play a role.

Second, the authors attribute the performance differences of pretrained LLMs across HMM tasks mainly to how much history the system needs to remember. But the two types of HMMs also differ in state transition structure, output distributions, and other aspects, so it cannot yet be concluded that memory depth is the decisive factor. One could measure the effective memory depth of the different HMMs and then examine its relationship with model performance.

Questions for authors

Questions for the authors:

  1. Can you provide complete proofs of Theorem 3 and Corollary 2, explicitly listing all required assumptions and every parameter affecting the complexity? If a full proof is currently out of reach, could the theoretical claims be appropriately narrowed?

  2. How should "system identification" and "interpolation" be rigorously defined? If a prediction method can be written both as parameter estimation and as kernel regression, how should it be classified? What evidence in the current experiments could genuinely rule out the possibility that the model uses partial identification or a mixed strategy?

  3. Can you directly quantify the effective memory depth of the different HMMs and verify whether it actually explains the performance differences of pretrained LLMs, rather than other system characteristics?

  4. Given the limited explanatory power of the condition number for per-system error, and that some preregistered HMM experiments did not meet the expected criteria, should the paper's overall conclusions be appropriately tempered?

Overall assessment: Borderline Reject (5/10)

I believe this paper contains publishable core findings and offers a valuable theoretical perspective for studying in-context learning. By comparing different prediction methods, the authors show quite clearly that a specially trained Transformer can exploit information to achieve good predictions even when the history is insufficient for the specific DeePC method to predict accurately. My evaluation is neutral at present, mainly because:

  • The important theoretical results for discrete sequences are not yet adequately proven.
  • Although the experiments reveal differences in prediction behavior across models, they are insufficient to directly pin down the internal computational mechanism, much less to establish general capability boundaries for pretrained LLMs.

If the authors can complete or appropriately narrow the discrete-sequence theory, and more carefully distinguish experimental observation from mechanistic inference, I would be willing to raise my score. Overall, the proofs need to be more complete, and every claim should be matched to the actual strength of the supporting evidence.

3 comments
AuthorsOpenReview ↗

Response to Reviewer aALS (1/2): W1 & Q1

Thank you for a review that located exactly where the claims outran the evidence. Your points are of three kinds: a gap in the proofs (W1/Q1), a distinction drawn too sharply (W2/Q2), and interpretations beyond the data (W3/Q3, Q4). We narrowed the first, redefined the second, and measured or withdrew the third. This part covers W1/Q1; part 2 the rest.

Reply to W1 & Q1 (Theorem 3 and Corollary 2). You are right on all three counts: HKZ's Condition 1 needs the single-symbol emission matrix to have rank r, impossible for r = 4 with 3 symbols; their Condition 3 (positivity) is needed for the KL guarantee; and nothing in the cited results connects a single trajectory to their i.i.d. sample size. A complete proof was out of reach in the discussion period, so we narrowed the claims as you suggested (App. A, "Discrete sequences"):

  • Assumptions, explicit. (A1) at a block length b with ∣Σ∣b≥r|\Sigma|^b \ge r, the blocked emission matrix has rank r with margins σO\sigma_O (emissions) and σ0\sigma_0 (blocked pair matrix); (A2) stationarity and (C,θ)(C,\theta)-geometric mixing (Balle & Maillard 2017); (A3) HKZ's positivity with constant α0\alpha_0.
  • Theorem 3 keeps only what is rigorous: the posterior predictive is the Bayes act for log loss, so its π\pi-averaged excess KL is at most that of any context-measurable estimator.
  • Corollary 2, labelled a proof sketch, states the rate under (A1)-(A3) with HKZ's sample size reproduced verbatim: a context length T0T_0 polynomial in b, ∣Σ∣b|\Sigma|^b, C/(θ(1−θ))C/(\theta(1-\theta)) and NHKZ(ε/2,η;r,∣Σ∣b,σO,σ0,α0b)N_{\mathrm{HKZ}}(\varepsilon/2,\eta; r, |\Sigma|^b, \sigma_O, \sigma_0, \alpha_0^b). Its three steps: blocking and alignment; Balle-Maillard's single-trajectory Hankel concentration in place of HKZ's i.i.d. Lemma 8 (their Lemmas 9-17 are deterministic); uniform smoothing at weight ε/8\varepsilon/8. It names the three steps it does not carry out: decoupling the evaluated history from the estimation prefix, a Frobenius-norm step in Lemma 8, and the discarded prefix. The claim is the polynomial form of T0T_0; the contributions and the Discussion list it as sketched, not proven.
  • The old Corollary 2 is split. Corollary 3 is rigorous: a predictor reading only the last d symbols has excess KL at least βd\beta_d, the truncation bias (a conditional mutual information, via the compensation identity), so it needs d≥τ(ε)d \ge \tau(\varepsilon). The count-based "exponential" argument is now a Remark labelled heuristic; no lower bound is claimed from it.
  • Assumptions measured on the classes we use (App. D, one table produced by one script):
Check on the cyclic class (8,400 sampled sources)Result
single-symbol emission rank = r for r = 4, 3 symbolsnever (rank at most 3), as you noted
blocked emission rank (b = 2) = r100%
Hankel rank = r at basis length 3100% in six of seven class-rank rows, 99.7% in one (LLM grid, r = 2)
relative margin σ0\sigma_0medians 0.11-0.46 across classes; 1-3% of sharp-emission sources below 1e-6
mixing, second eigenvalueabout 0.94
positivity α0\alpha_0holds, median about 1e-5: the constant in the rate is large for sharp emissions

We state the small constants rather than hide them. Part 2 follows.

AuthorsOpenReview ↗

Response to Reviewer aALS (2/2): W2 & Q2, W3 & Q3, Q4

Reply to W2 & Q2 (interpolation vs. identification). Agreed: least squares in its dual form is kernel regression, and a kernel-shaped prediction does not show that no rule was learned. The revision defines the two notions by the information each needs, not by algebraic form (Sec. 3, "Two reference predictors, two regimes"): the behavioral predictor recombines windows of the prompt and is exact once they span the behavior (k≥kbehk \ge k_{\mathrm{beh}}); the identifying predictor (ARX at the true order, or the Bayes predictor) is exact once the prompt determines the system (k≥kidk \ge k_{\mathrm{id}}). A trained model is "in the identification regime" if its error collapses far before kbehk_{\mathrm{beh}} and it passes the fresh-probe test, "in the interpolation regime" if its error stays at the behavioral reference and does not fall with informativity, and "information-gated" if it falls with informativity but fresh probes are unavailable. "Mechanism" is replaced by "regime" throughout. On what the experiments rule out: nothing about the internal computation, and the text now says so. We added the measurement you asked about: by the gap to each reference prediction, LLM answers sit midway between 1-NN and OLS (relative position 0.42-0.48 across eleven models; 0.46-0.48 between the NW kernel and ARX on continuous prompts), so the probes fix an error level that is flat in prompt information, consistent with but not proof of kernel smoothing; partial identification or a mixed strategy is not excluded. The Discussion separates proven, observed and conjectured statements, and marks the conjectures.

Reply to W3 & Q3 (training data; memory depth). Training data: the claim is now "the training prior selects the regime at a fixed architecture", with the sentence that capacity, architecture and optimization were held fixed, not shown irrelevant. Memory depth: we did what you proposed. For every HMM source the LLMs were run on we computed the truncation bias βd=E KL(f(⋅∣x1:T) ∥ f(⋅∣xT−d+1:T))\beta_d = \mathbb{E}\,\mathrm{KL}(f(\cdot\mid x_{1:T}) \,\|\, f(\cdot\mid x_{T-d+1:T})) and the depth τ(0.01)=min⁡{d:βd≤0.01}\tau(0.01) = \min\{d : \beta_d \le 0.01\} (App. C, two new figures):

within the cyclic classacross the two classes
memory depth vs. LLM excess KLSpearman 0.63-0.86 for six of seven models; partial correlation +0.32 to +0.89 given mixing and entropy (five of six significant)depth ranges overlap completely (cyclic 1-8 symbols, lazy-chain up to 16); the deepest-memory lazy-chain cells are handled well
conclusionmemory depth organizes error inside a classit is not the boundary between the classes

The paragraph "the boundary is latent memory depth" is withdrawn and replaced by "Two HMM classes, and the measured memory depth"; the classes are named by construction (lazy-chain / cyclic); the unexplained class difference (cyclic sources carry higher error, 0.067 vs. 0.045 nats, without beating the n-gram) is listed as a limitation; an encoding control (letters vs. digits) changes OPT's KL by at most 20%, so encoding is not the cause.

Reply to Q4 (tempering). Yes. The abstract, contributions, Sec. 5 and the conclusion were restated: they no longer claim a knee "at" kidk_{\mathrm{id}} (see our reply to Reviewer Wau2: the error reaches its floor two to three tones later) nor that memory depth is the boundary; every mechanism claim is now a regime claim defined by a measurable threshold; the discrete rate is labelled a sketch; and the Discussion separates proven, observed and conjectured statements.

These changes follow your two conditions: the discrete theory is narrowed with every assumption listed, and observation is separated from mechanism. We would be grateful if you would reconsider the score in their light.

Thank you again for your review. Since the discussion closes on November 18, we would be grateful to know whether the revision meets the two conditions you set. The discrete theory is now stated under explicit assumptions (A1)-(A3), with a complete proof of Corollary 2 (App. A: two new lemmas and a six-step proof that covers blocked observations, single-trajectory estimation, and the step from estimated statistics to predicted probabilities). Observation is separated from mechanism (Sec. 3, "Two reference predictors, two regimes"; the proven / observed / conjectured paragraph of the Discussion). If any step of the proof is unclear, we will answer before the deadline.

Comments on the paper

1 comment

We thank the three reviewers. Unchanged: the thesis (two information thresholds, kid=⌈nx/2⌉k_{\mathrm{id}}=\lceil n_x/2\rceil for identification vs. kbeh=⌈L/2⌉k_{\mathrm{beh}}=\lceil L/2\rceil for behavioral prediction), every experiment and every data point. Changed: the precision of the claims, the assumptions made explicit, one new control experiment, and the presentation. Main text stays at 10 pages. The table maps each concern to its change; the per-reviewer replies use the reviewers' own numbering.

ConcernChangeWhere in the revision
aALS W1/Q1: proofs of Thm 3 / Cor 2 incompleteassumptions (A1)-(A3) stated; Thm 3 reduced to the rigorous Bayes reduction; the rate is Cor 2, labelled a proof sketch with its three open steps named; Cor 3 is a rigorous truncation-bias bound; the "exponential" count argument is a Remark marked heuristic; assumptions checked numericallyApp. A "Discrete sequences"; App. D table "Assumptions (A1)-(A3), measured"
Wau2: knee location in Fig. 1two estimators (exact collapse; broken-stick) with paired bootstrap intervals, applied to every predictor; "k = 2-3" withdrawn; new rollout-horizon sweep N = 4-16Secs. 5.1-5.2; App. D "Knee estimation", figures "Systematic knee estimates" and "Rollout-horizon sweep", knee table
aALS W2/Q2: interpolation vs. identification too sharpregimes defined by the threshold at which error collapses and by the fresh-probe test, not by algebraic form; "mechanism" replaced by "regime"; LLM position between the two references (0.42-0.48) reported; proven / observed / conjectured separatedSec. 3 "Two reference predictors, two regimes"; Sec. 5.5; Discussion
aALS W3/Q3: memory depth asserted, not measuredtruncation bias and memory depth computed for every HMM source; it organizes error within the cyclic class, not across classes; boundary claim withdrawn; classes renamed lazy-chain / cyclicSec. 5.5 "Two HMM classes, and the measured memory depth"; App. C, two figures
aALS W3: "training data fully determines the mechanism"scoped to "the training prior selects the regime at a fixed architecture"App. D, table "training prior selects the regime"
aALS Q4: temper the conclusionsabstract, contributions, Sec. 5 and conclusion restated; mechanism claims become regime claims; the discrete rate labelled a sketch; proven / observed / conjectured separatedAbstract; Secs. 1, 5, 6, 7
q3JD 1-3: formality, headers, abstract, intro, conclusion, experiment specification, citationsEq. (1) fully typed; proof sketches after Props. 1-2 and Thm. 1; neutral headers; abstract rewritten; plain-language opening; Protocol paragraph with hypotheses H1-H5; per-experiment table; 7 model citations; 21 references on ICL and its limits and on data-driven controlSecs. 1-5; App. D table "Per-experiment summary"
q3JD question: positioningrelated work states, for each closest work, the axis on which ours differs; limitations listedSec. 2; Discussion

Three statements are withdrawn because the evidence did not support them: the knee "at" kidk_{\mathrm{id}} (the error reaches its floor two to three tones later), memory depth as the boundary between the two HMM classes, and the discrete-sequence rate as a theorem. Where a claim is withdrawn, the revision says so.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.