acceptodds
Under review as a conference paper at ICLR 2027

In-Context Learning as Query-Conditioned Hypothesis Selection

Abstract

In-Context Learning (ICL) is the ability of Large Language Models (LLMs) to alter their input–output mapping by conditioning on demonstrations in the prompt, without parameter updates. Beyond operational differences, ICL is different from Empirical Risk Minimization in the generalization behavior, calling for distinct explanatory frameworks. Theoretical accounts explain ICL as different forms of adaptation to the demonstrations, ranging from Bayesian inference over hypotheses given the demonstrations, to implicit execution of a learning algorithm that fits a predictor to them. We propose modeling ICL as case-based, transductive adaptation in which LLM predictions reflect query-conditioned switching among predictive modes, rather than executing a fixed implicit learner. We formulate this theoretically and test in controlled regression tasks by analyzing predicted distributions for branch structure (multiple modes), discovering query-dependent predictive mass shifts and the causal verification of modes through activation steering. This perspective reconciles disparate theories of ICL by interpreting different implicit learner interpretations as accounts of behavior at query-demonstration regimes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
In-context learning, Large Language Models, Transductive learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wQkZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates In-Context Learning (ICL) in large language models through controlled one-dimensional scalar regression tasks. Drawing on the human function-learning literature, the authors propose a view of ICL as query-conditioned hypothesis selection: demonstrations induce a repertoire of candidate predictive modes, while the query determines which mode is expressed. This perspective aims to extend accounts based on fixed implicit learning algorithms or demonstration-conditioned hypothesis mixtures.

The authors evaluate this hypothesis using eight open-weight autoregressive language models across ten synthetic data-generating functions, five demonstration densities, and five interpolation/extrapolation regimes. Their methodology combines several complementary analyses:

  1. Prediction-surface analysis: Sequential RANSAC peeling with simple function primitives (constants, lines, and low-degree polynomials) identifies branches in greedy prediction surfaces. Local interleaving tests and nonparametric modality tests across independently sampled demonstration sets provide additional evidence of predictive multiplicity.
  2. Activation steering: Interventions on intermediate residual-stream representations at the final input-token position test whether predictions on held-out queries can be redirected between identified branches.
  3. Pointwise attribution: Predictions are compared against a library of 34 classical estimators, grouped into five inductive-bias families (Linear, Polynomial, Tree, Kernel, and Periodic), under global and local fitting scopes.
  4. Output-probability analysis: Token-prefix expansion provides bounds on numerical-answer probabilities, enabling tests of whether fixed hypothesis-mixture weights can explain the observed query-dependent behavior under specified concentration assumptions.
  5. Controlled pretraining: Small GPT-2-like transformers trained on known linear and mixed function distributions are used to examine how pretraining support influences the emergence of multimodal predictive behavior.
Critical strengths

1. Complementary Methodological Approaches to Predictive Multiplicity

Rather than relying solely on aggregate prediction errors or visual inspection, the authors combine several complementary probes of ICL behavior: RANSAC-based surface decomposition, local branch-interleaving tests with matched single-function controls, nonparametric modality tests across demonstration sets, numerical-answer probability analysis, and causal activation interventions. In particular, the local branch tests and held-out steering experiments provide meaningful evidence beyond post-hoc curve fitting. This methodological breadth strengthens the paper's behavioral characterization of extrapolation in scalar-regression settings.

2. Conceptual Motivation and Transparent Reporting of Limitations

The connection to classical human function-learning research, including knowledge partitioning and hybrid rule–exemplar accounts, provides an interesting conceptual perspective on ICL. The authors also report several findings that qualify their central hypothesis, including substantial ambiguity in classical-predictor attribution (Table 11), sensitivity of probability-based conclusions to numerical acceptance intervals (Appendix J), and the limited emergence of multimodal behavior in transformers trained on controlled function distributions (Table 14). These analyses make the empirical scope and limitations of the proposed interpretation more transparent.

Critical weaknesses
  1. Internal Hypotheses vs. Numerical Decoding Artifacts While the local interleaving tests (Appendix I, Table 12) rule out simple continuous noise, interpreting prediction branches as distinct internal hypotheses remains unestablished:
  • Decoding Artifacts: Autoregressive float generation introduces discrete effects (quantization, rounding, token-frequency bias) not captured by continuous baselines. Constant output branches (e.g., Figure 3) may simply reflect numerical token attractors rather than algorithmic hypothesis switching.
  • Steering Specificity: Activation steering (Appendix E, Table 10) uses output-defined vectors with large perturbations (up to 4x mean difference) but lacks norm-matched random/orthogonal controls or magnitude-matched donor groups. This demonstrates token-level steerability rather than mediation of an abstract hypothesis-selection mechanism.
  1. Fragility, Sensitivity, and Narrow Scope of Reweighting Evidence The probability-based evidence for query-conditioned reweighting (Section 4.2, Appendix J) is highly brittle and conceptually restricted:
  • High Sensitivity: Only 3 of 30 model–task pairs show certified reweighting and dominant switching at n=100. Expanding the acceptance band from 2% to 10% alters positive classifications from 2/60 to 17/60, showing heavy dependence on arbitrary thresholds.
  • Restricted Mixture Test: The rejection applies only to candidates concentrating >=95% probability mass in designated bands (epsilon=0.05), not general fixed-weight Bayesian mixtures whose predictive modes naturally shift with queries.
  • Attribution Ambiguity & Selection Bias: 61–65% of queries have ambiguous estimator attribution (Table 11). Furthermore, DeepSeek-R1 exhibits ~46–48% parse failures at n >= 80 (Table 2); dropping these introduces severe survival bias.
  • Limited Generality: Controlled-pretraining tests (Appendix K, Table 14) rarely exhibit multimodal switching, indicating that the phenomenon is heavily contingent on specific 1D regression setups rather than a general property of ICL.
Questions for authors

1. Robustness to Numerical Representation and Decoding Controls

Do the recovered branches and local interleaving patterns persist when numerical formatting is modified, such as through changes in decimal precision, scientific notation, or rescaling outputs to integer-valued ranges? Can the authors evaluate the branch detector against controls that explicitly reproduce discrete numerical tokenization, quantization, or autoregressive decoding effects, rather than only continuous-noise perturbations?

2. Specificity of Activation-Steering Interventions

How do the reported induction and removal rates compare with those obtained using (a) random or orthogonal steering vectors of matched Euclidean norm and (b) donor groups matched for predicted output magnitude and query location? How large is the applied perturbation relative to the residual-stream norm at the selected layer? These controls would help establish whether the intervention specifically targets predictive hypotheses rather than more general output-correlated directions.

3. Robustness and Scope of Probability-Based Reweighting

Given that only 3 of 30 model–target-function combinations satisfy the joint reweighting and dominant-switching criterion at 100 demonstrations, how stable are these certificates under alternative candidate discovery sets and numerical acceptance-band definitions? Which classes of fixed-weight mixture models are excluded by the test, and which remain compatible with the observations under the stated 95% concentration assumption? Can the authors provide additional evidence that the conclusion is not primarily driven by the chosen candidate family or interval resolution?

4. Missing Predictions and Attribution Consistency

Can the authors report parsing success rates stratified jointly by model, target function, demonstration count, and extrapolation distance, and assess how excluding unparsed reasoning-model outputs affects the estimated distribution of predictive modes? Additionally, do the Linear-to-Periodic attribution trends in Figure 4 persist under tie-aware assignment using the same five-family taxonomy and estimator library as the main analysis, particularly given the high ambiguity rates reported in Table 11?

5 comments
AuthorsOpenReview ↗

1. Robustness to Numerical Representation and Decoding Controls

Thanks a lot for your thoughtful reviews. This helped us improve the paper.

1. Do the branches survive when we change how numbers are written?

Thank you for the question. We do the exact robustness check you suggested. Yes, the proposed explanation hold the robustness checks.

The number format changes both the values the model repeats and how often it switches between them. We tested Qwen2.5-7B and Gemma-3-12B on the same linear and cubic functions, writing the numbers with two decimals, four decimals, as rescaled integers, or in scientific notation. Both models still produced repeated output levels in every format. There is some interesting changes in behaviour about which numbers they repeated. On the linear task, Qwen repeatedly gave answers between 213 and 337 with two decimals. Those large values disappeared with integers and scientific notation. The overall analysis suggest that the hypothesis learnt from the demonstrations depend on the format the demonstrations are presented, and the query conditionining is the dominant explanation.

To measure switching, we looked at 30 nearby queries at a time and counted a window when the predictions alternated between two fitted lines. On the Qwen linear task, 8 of 40 windows were flagged with two decimals, 7 of 40 when we repeated that same run, 3 of 40 with integers and 14 of 40 with scientific notation. So the format matters even when the underlying regression problem stays the same.

Could decoding alone produce these flags? We now add this to our null and started with a single smooth curve and altered its printed answers using each model’s own number tokens. We tried rounding, cutting the number at a token boundary and changing its first token. At the rates measured in the models, these controls flagged at most 13.5% of extrapolation windows, compared with 15.7% for the five models tested.

Your suggestion about repeated numbers gave a different result. We let the smooth curve occasionally return a fixed value, such as zero or the largest demonstrated label. At the pooled frequency observed across six models, this control flagged 21.1% of windows, compared with 18.1% for the models. A flag alone therefore gives us too little reason to infer a second rule. The clearest exception was Qwen2.5-7B. When both fitted lines had a slope, it flagged 145/556 windows (26.1%), against 10.2% for its control. It was the only model with enough such windows to show an excess. The overall takeaway remains that the hypothesis learnt from the demonstrations depend on the format the demonstrations, and across formats query conditioned switching holds. We will add the full robustness check analysis in the appendix.

AuthorsOpenReview ↗

2. Specificity of Activation-Steering Interventions

2. Does steering move the model from one rule to another?

We rebuilt the steering experiment with nearly the same input value, then compared it with 10 random and 10 orthogonal directions of the same length. The mode steering vector direction moved 167/210 predictions away from previous expressed mode. The strongest control baseline moved only 29/210. This answers the query-location concern for that example. It still matters where those predictions went. With a narrow band around each fitted mode, 84 of 101 departures landed outside every band for the control across the board.

DirectionRemoval at α = −1Induction at α = +1
Original steering direction181/210 (86%)53/156 (34%)
Highest count among the 20 controls29/210 (13%)84/156 (53%)
Steering direction from 35 donor pairs matched on query location167/210 (80%)84/156 (53%)

(b) Matched donors. After matching on query location, removal still exceeded every control. The donors’ absolute output ranges did not overlap (0.96–73.25 versus 227.12–325.46), so we could not also match on magnitude.

Perturbation size. At |α| = 1, the Figure 3 perturbation was 17.3% of the median residual-stream norm at the selected layer. Across the four regenerated Table 10 experiments, it was 4–21% at |α| = 1 and 16–84% at |α| = 4. The Figure 3 controls support suppression of one constant answer. Tests of transfer between two sloped branches gave no consistent effect.

AuthorsOpenReview ↗

3. Robustness and Scope of Probability-Based Reweighting

3. How much does the probability result depend on the candidate rules?

It depends on both the rules we test and how close an answer must be to count as a match. For each query, we measure how much answer probability falls near each candidate’s prediction. We then ask whether the same weight for that candidate could explain two different queries. This allows a rule’s prediction to change with the input.

At 100 demonstrationsQwen2.5-7BGemma-3-12B
Demonstration sets showing a weight change, using candidates found on separate discovery queries39/5027/50
Same test, using five candidates specified before seeing model outputs29/5022/50

The full criterion also asks whether the highest-weight candidate changes. Three of 30 model–function combinations passed with discovered candidates. We repeated discovery on five fresh query grids. Under the new test, the full result held on 6/6 grids forall three Qwen sinusoid, Qwen piecewise-linear Gemma exponential, counting the original grid. Changing the band width also mattered. With the candidate list held fixed, all of the three combinations passed at all three widths tested (2%, 5% and 10%) on the new seed run we executed.

Your question also exposed a shortcoming the old test could call it dominant even if the remaining candidates together held more probability. We now require the winner’s lower bound to exceed the upper bound for every competing group. This removed the false switches in our fixed-weight simulations. The three combinations still pass at the original 5% width. We will restate the claim in the main text.

Reviewer_B2LkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Large language models can perform new tasks without updating parameters by using only a few prompt demonstrations, a behavior called in-context learning (ICL). Existing theories assume models extract a single rule from demonstrations and apply it globally. This paper challenges that assumption. The authors argue that demonstrations activate multiple candidate rules simultaneously, and the model selects among them based on the query position itself. They term this mechanism "query-conditioned hypothesis selection."

To test this idea, the authors evaluate eight open-source models on ten known function classes using fixed demonstrations and query inputs that range from in-distribution to far out-of-distribution. They analyze the results across five dimensions:

  1. Piecewise curve fitting to check if predictions fragment and oscillate between lines.
  2. Activation steering to test if the model can be guided between prediction modes.
  3. Comparisons with classical regression algorithms across different input regions.
  4. Probability analysis to verify if fixed mixture weights explain predictions across all positions.
  5. Controlled pretraining on synthetic data to isolate demonstration support from training distribution effects.

The results confirm that predictions fragment systematically rather than randomly, activation interventions reliably steer modes, matched algorithms vary by region, and fixed weights fail to explain the distributions. Thus, jagged ICL behavior stems from query-dependent mode switching, suggesting that minimal input-level supervision at inference time can identify and steer the desired prediction rule.

Critical strengths

1.Clean separation of query-induced shifts. By fixing demonstrations and varying only query positions, the design separates query-dependent behavior from demonstration-dependent shifts, which is uncommon in current ICL work. The local branching test in Appendix I is solid: it requires predictions on adjacent queries to switch between two distinct lines at least twice, calibrated against five baselines: Gaussian noise, heavy-tailed noise, single smooth curves, saturation curves, and synthetic ground-truth branches (Table 12). Models show branching rates of 15.5% to 19.4% in extrapolation regions, while all single-curve controls remain below 1.3% and planted-branch controls exceed 90%. This confirms piecewise structure that noise cannot explain. This is the paper's most reliable finding and the main reason to avoid a clear rejection.

2.Transparent reporting in appendices. The authors honestly document unfavorable outcomes: Llama-3.1-8B yields 0/50 on the probability test (Table 13), synthetic small models exhibit almost no multimodality (Table 14), attribution patterns are matched by a single predictor with additive noise (Table 11), the probability test is sensitive to interval width (positives jump from 2/60 to 17/60 when widening from 2% to 10%), and DeepSeek suffers high parse failure rates (Table 2). These details allow reviewers to gauge evidence strength and offer grounded targets for narrowing claims in revision.

3.Comprehensive scope and reproducible setup. The evaluation spans eight models, ten function classes, five demonstration densities, and thirty peeling configurations. Prompt templates, parsing rules, seed derivations, hyperparameters, and compute budgets are fully documented (Appendices B, D, H).

Critical weaknesses
  1. Identified modes lack substantive richness. Observed modes mostly alternate between linear continuation and constant collapse, which fails to support true hypothesis selection. In Figure 3, Qwen2.5-7B yields an affine trend and two constants (near 30 and -277); the authors claim 30 reflects the demonstration mean, but the true mean on [−10,10][-10, 10] is -3. Appendix I notes that nearly half of extrapolation windows consist of two flat levels. Extrapolating from shifts between linear trends and plateaus to unifying algorithmic, Bayesian, and kernel theories overreaches. Furthermore, continuous-input Transformers in Appendix K reject unimodality in only 4 of 1,296 tests, showing that discrete switching disappears without tokenization artifacts—a confound the paper leaves unaddressed.

  2. Fragile probability tests and post-hoc candidate families. Peeling and attribution only capture greedy prediction geometry and cannot prove shifts in hypothesis weights without a pre-specified candidate set. Here, candidate families are mined post-hoc from 25 discovery queries and shift when those queries change. The test results are weak: Llama is excluded due to high unparsed probability mass (0.568); only 3 of 30 model-function pairs show both reweighting and dominant-mode switching; and certified coexistence is marginal (8.9% on Qwen, 2.2% on Gemma). The reported 63% and 35% rates are conditional on queries where greedy predictions already land in different bins.

  3. Mismatched tolerance scale in global peeling. Algorithm 1 scales inlier tolerance to output range. For exponential functions, true values reach 102210^{22}, creating an enormous tolerance where B=1B=1 across all configs (Table 6). In contrast, bounded model outputs force a narrow tolerance, yielding 7 to 9 modes and an artificial Δ\Delta of +5.2 (Table 7). Using positive Δ\Delta values on fast-growing functions to claim excessive fragmentation is therefore invalid. Bounded targets like sinusoids produce the opposite distortion, where outliers widen tolerance and suppress mode counts. Valid comparisons require uniform absolute tolerances.

  4. Weak causal link in activation steering. Quantitative steering is tested only on affine functions across four models, while the cubic case in Figure 1 is purely schematic. Donor selection does not match query ranges, so steering vectors likely encode query magnitude rather than hypothesis representations. The protocol lacks random-direction and scale-matched baselines. Moreover, evaluation thresholds count any shift past the mode midpoint as success, and the removal rate counts transitions into unclassifiable space. This confirms output perturbation, not hypothesis manipulation.

  5. Discrepancies between main text and appendix evidence. The main text overstates empirical backing. Appendix G reveals that 61% to 65% of query attributions are indistinguishable and match a single predictor with additive noise within 0.05. The claim that multimodality occurs within training range conceals that only 4 of 1,296 tests in Table 14 reject unimodality, all clustered around one seed. Section 4.1 claims evaluation across all δ\delta at 5% tolerance, but Table 4 restricts queries to extrapolation (δ>0\delta > 0) and Table 1 uses 1.5%, 3%, and 6%. Finally, up to 48% of DeepSeek-R1 queries fail to parse, leaving zero valid windows for branching analysis, yet the main text presents branching as an eight-model aggregate without disclosing this omission.

Questions for authors

Responses to the following questions will affect my score.

  1. Stripping tolerance scaling (corresponding to Weakness 3). If the same absolute tolerance is applied to both ground truth functions and model predictions, for example scaled by the exemplar output range or the local window range, what do BB and Δ\Delta become for exponential, quadratic, and cubic cases in Tables 1, 6, and 7? Please also report the interior point coverage of the ground truth functions under each configuration, to show what fraction of points are not absorbed by any segment when BB equals 1.
  2. Baselines for steering experiments (corresponding to Weakness 4). Please provide three additional sets of results: norm-matched random directions; directions computed within the same mode grouped by query magnitude, to test whether steering vectors merely encode query values; and induction and removal rates when using the original stripping tolerance τ\tau rather than half the inter-mode distance as the classification bandwidth. In addition, please provide the full transition matrix, including outputs falling into unclassified regions.
  3. Stability of candidate hypothesis families (corresponding to Weakness 2). How did the affine candidate family for Qwen change after using a different set of discovery queries? Under this new candidate family, do the certification results for reweighting and dominance switching still hold? Can a prospective validation be conducted on at least one new exemplar set using a predefined candidate family, rather than mining candidates from the same surface?
  4. Classification of degenerate modes (corresponding to Weakness 1). In Figure 3, M2 is described as "collapsing to exemplar mean," but the affine mean on [−10,10][-10, 10] is −3-3, whereas M2 is around 30. What do M2 and M3 actually represent? If constant plateau modes are removed from the analysis, retaining only modes with non-zero slopes or non-linear shapes, what do the local branching rates in Table 12 and the certified proportions in Appendix J become?
  5. Interpretation of Appendix K. The T-mix model yields predictions falling between two continuations under ambiguous exemplars, rejecting unimodality only 4 times out of 1,296 tests. How do the authors reconcile this result with the framework where "exemplars induce multiple modes and the query selects one"? Do the authors believe that the discrete branching observed in pretrained language models depends on numerical tokenization? If so, what does this imply for the generality of the proposed framework?

These following are clarifying questions and do not directly affect my score. 6. Section 4.1 states that stripping is "evaluated across all δ\delta intervals" with a 5% tolerance; Table 4 includes only δ>0\delta > 0, while Table 1 uses 1.5%, 3%, and 6%. Which of these is the actual configuration? 7. Do the summary results for local branching in Section 4.1 include DeepSeek-R1? If this model lacks evaluable windows in Appendix I, how should the description of "eight models" in the main text be updated? 8. GPT-OSS-20B is a sparse MoE model, whereas Section 4 excludes MoE models from the primary conclusions on the grounds that MoE routing mechanisms trivially explain query-dependent mode switching. Does using it as the representative model in Figure 2 and describing it as a "3.6B fixed model" contradict this exclusion rationale?

4 comments
AuthorsOpenReview ↗

A natural language task to assign semantic meaning to the hypothesis learnt in context

Thank you for the thoughtful review and pointing out the errors and places our measurements could improve. We have done the analysis you asked for and will add this ot the paper. Before we answer your questions we present a new semantic experiment which we believe address the broad weknaess you allude to.

A natural languge task to assign semantic meaning to the hypothesis learnt in context

To show that LLMs indeed learn meaningful hypothesys and switch between them based on queries, we use two new natural language 'like' datasets. Just like the regression task, LLM is given x->y pairs that could be explained by some rules and then asked to predict for a new x (x->?), where x and y have semantic meaning.

(1) Country task.\color{teal}{\text{Country task.}} We give pair of country and city for the LLM to learn from. In the demonstrations are pairs for which x is a county and y is both the capital was also the largest city. Both rules therefore fit every example. We then query 29 countries for which the rules (capital and largest) give different answers.

We asked the same 29 country questions with 10 different sets of demonstrations. This let us ask the two questions. Did the model hypothesis depend on the demonstration or query? We evaluate this on four different models and the results are:

ResultAcross the four models
Demonstration sets in which each model gave both kinds of answer20/20
Variance in capital-answer probability explained by query identity80.2–98.4%
Variance explained by the demonstration setAt most 1%
Answers naming the largest city40–105 of 580 per model

We compared countries by averaging across demonstration sets and compared sets by averaging across countries. The ranges cover four models, each tested with 10 and 20 examples per prompt. We computed these shares separately for each model and demonstration count. For the country effect, we averaged each country’s capital probability over 10 demonstration sets and compared the 29 country averages. For the demonstration-set effect, we averaged over the same 29 countries and compared the 10 set averages. Capital probability was normalized over the candidate city names for each query.

Capitals remained the dominant as one would expect. The largest-city/economic center was different from capital for all the query 29 countries. We then do steering and show that we can indeed show both the hypothesis at every query with an accuracy of 86% across the models. In short the depending on query LLMs used both hte hypothesis and they were steerable awith the same method as the paper.

Element task.\color{teal}{\text{Element task.}} We gave name of an element from the periodic table as x and the corresponding y for demonstations satisfy (a)“the chemical symbol” and “the first letter of the name” (example Hydrogen-> H). On transductive querying for elements where the rule split, most models learnt rule (a) as expected 1,541 of 1,590 answers. But even in this setting a minority showed the rule (b). And this sample (80/20 split for constructing vector and evaluating) was enough to steer models into hypothesis (b) with an accuracy of 82%.

The evidence supports query-dependent expression of a rule in context.

AuthorsOpenReview ↗

Ground truth use the same tolerance, and can query position alone produce the steering effect

Q1. What happens when the model and the ground truth use the same tolerance?

As your intution points to, the same percentage can give very different tolerances. In the submitted analysis, each fit used a percentage of the output range of the points remaining in its own pool. On the exponential task, one very large ground-truth value can make that band wide enough to absorb much of the curve. The model’s predictions can occupy a much smaller range. The reported worst case difference of +5.2 (in Appendix D, Table 7) branches therefore cannot, on its own, establish that the model follows more rules.

We repeated the comparison using one absolute tolerance for both curves, set to 3% of the demonstration output range. The table shows the line-only fit at 100 demonstrations, averaged over the eight models for comparison with Table 1. Coverage is the fraction of ground-truth query points assigned to any fitted line.

TargetGround-truth linesMean model linesDifferenceGround-truth coverage
Exponential24.6+4.632.5% (as expected the lowest)
Quadratic24.0+2.076.3%
Cubic23.6+1.674%
Affine13.4+2.4100% (for comparison)

Why report coverage? When we allowed a quadratic fit before fitting lines, all three targets returned one ground-truth segment. That segment covered 90% of the exponential points, 100% of the quadratic points and 84% of the cubic points. The same count of one has a different meaning in each case. We will give coverage alongside the counts and remove the inference about exponential over-fragmentation.

Q2. Can query position alone produce the steering effect? Where do the predictions go?

We split queries from the same starting mode into larger and smaller input values and subtracted their mean activations. We tested this direction on held-out queries and compared it with the between-mode direction and 20 directions of the same length at the same layer.

Held-out queries leaving the starting modeWithin-mode directionBetween-mode directionLargest of 20 controls
Qwen2.5-7B, α = +113/8937/89 [Qwen departures exceed the within-mode control (13/89) and the best random or orthogonal control (8/89). At most 13/89 would lose the advantage over query position.]8/89
Llama-3.1-8B, α = −262/45088/450 [Llama departures exceed the within-mode control (62/450). At most 62/450 would lose that advantage, even if the direction still beat the random controls.]5/450

The within-mode direction did move predictions. Of those departures, only 1/13 in Qwen and 4/62 in Llama reached another identified mode. [Few query-position departures reached another branch.] The others landed outside the fitted bands.

In the Figure 3 experiment we also built a direction from 35 donor pairs matched on query location. At α = −1 it moved 167/210 predictions out of the target mode, compared with at most 29/210 for the ten random and ten orthogonal controls. [After matching query location, the direction suppressed the plateau on 167 of 210 queries. At most 29/210 would put removal within the observed control range.] In the reverse direction, at α = +1, it moved 84/156 into the target mode, far above to the best control. The two donor groups had disjoint output-magnitude ranges, so the comparison controls for query location while leaving output magnitude open. We find evidence for suppressing this plateau.

We also rescored the original Figure 3 outputs at the peeling tolerance (166 output units in this run). At α = −1, 214/225 predictions left the target band. At α = +1, 58/203 entered the target band. These figures describe a separate rescoring of the original run; its 460-prompt pool included 35 prompts used on one side of the donor contrast. The matched-donor comparison above is the relevant control for held-out transfer.

AuthorsOpenReview ↗

Change of candidate rules and branching , constant modes

Q3. Do the probability results survive a change of candidate rules?

At 100 demonstrations, we found a change in candidate weight in 39/50 Qwen2.5-7B demonstration sets and 27/50 Gemma-3-12B sets using candidates discovered on separate queries. With five rules specified before seeing the outputs, those counts were 29/50 and 22/50. These existing sets still certify reweighting with predefined rules. The 63% and 35% in the submission use a different denominator: eligible query pairs whose greedy answers already landed in different bands.

For the Qwen affine example you asked about, the first two discovery grids both yielded a line with slope close to three, but its fit moved from 3.04x − 3.6 to 3.09x + 2.3. The repeated constants also changed. Across seven grids for this demonstration set, six gave evidence of reweighting and four gave a change in the dominant candidate. This analysis adds more evidence to the central claim.

For the three model–function combinations that passed the paper’s stronger criterion, we repeated discovery on five fresh grids. Counting the original grid, the result held on 6/6 for Qwen sinusoid, Qwen plinear and Gemma exponential. Overall, for rule switching has almost the same repeatability with two nonconstant candidates. We also adjusted the dominance test so that a winner’s lower bound must exceed the upper bound of every other group, including groups of overlapping candidates. This removed false switches in our fixed-weight simulations. All three combinations still passed at the original 5% band width.

Your prospective test asks a further question: can rules fixed in advance predict a weight change on a new demonstration set? With one new set for each of six model–function combinations, the five predefined rules certified reweighting in 5/6 cases. An extended family certified it all 6/6.

Q4. What are the Figure 3 plateaus, and how much branching remains without constants?

The description of M2 as the demonstration mean was wrong, we now correct it. For the stated affine rule on [−10, 10], even the largest possible demonstration output is 22. In the Figure 3 experiment, we evaluated 500 queries. On 51 it copied the first demonstration answer, −27.74, exactly. Another 228 predictions clustered around −273.45, far from every demonstrated answer, we now call it degenerate modes as suggested by another reviewer. 43 fell within 50 units of the true line, one of the three hyothesis discovered in the illustrative example. We will correct the caption and leave the origin of the negative plateau open.

How often was a flat branch degenerate? We checked whether its level matched the output of the nearest demonstration or the mean of the demonstrated outputs. Both rules can give the same answer across several nearby queries. We treated these matches as interpretable fallbacks and excluded them from the degenerate count. Under this stricter definition, the reported degenerate share was 18.6% For Qwen2.5-7B, 145/556 windows with two sloped fits were flagged 26.1%, compared with 10.2% for its control. We have added a full analysis on degenerate modes in appendix and mentioned this in main text.

In Appendix J, when the weight change had to be carried by a nonconstant candidate, the counts were 22/50 for Qwen and 21/50 for Gemma. These sets certify a weight change for a nonconstant candidate. When the dominant candidate had to switch between two non constant rules, the counts were 8/50 and 9/50.

AuthorsOpenReview ↗

numeric tokenization, deepseek and gptoss clarification

Q5. What does Appendix K tell us about switching and numerical tokenization?

The small transformers trained on lines and sinusoids generally predicted values between the two continuations. The four detections among 1,296 tests occurred at adjacent query locations in one training seed. Each test fixed the query and redrew the demonstrations. For validity we three seeds now.

To isolate the readout in the small-model experiment, we trained the mixed-function model to assign probabilities to 2,000 output bins. Reading out the mean of that distribution gave 201/648 detections across demonstration redraws. This ~31% of the tested settings showing evidence that the demonstrations change the prediction enough for the difference to survive averaging. Choosing the most probable bin from the same trained networks gave 482/648.

We tested numerical formatting directly in Qwen2.5-7B and Gemma-3-12B on the affine and cubic problems. Both produced repeated levels with two decimals, four decimals, rescaled integers and scientific notation. We have done this robustness check for wQkZ in detail. The values and switching rates changed but the observation is robust.

We have added a new experiment which might solve multiple questions in this direction on the robustness of finding using a new semantic task. After answering your questions we post that as a seperate comment. Please take a look.

Q6. Which settings produced the peeling tables?

Table 1 labels its three line-only rows 1.5%, 3% and 6%. Table 4 specifies extrapolation queries only (δ > 0) for the global peel. Section 4.1 these rows as using 5% across all distance bins, we now resolve this.

Following your question we-reran the analysis. Our reconstruction indicates that the printed Tables 1, 6 and 7 use extrapolation alone for the model prediction and ground truth}}. We will use the same extrapolation queries on both sides and state the tolerance beside each result. Thanks for helping us clarify this.

Q7. Did DeepSeek contribute to the local branching result?

We understand the confusion, Deepseek was previously not involved in all the anlysis as the reasoning trace sometime terminated and then analysing the next otken is ambigious. This caused confusion in the reporting in the paper. We now correct that. In the separate rerun, 8,578 of 12,500 completions finished with a numeric answer, four finished without one, and 3,918 ended during reasoning. The branching rate was 24.3 on finished numeric answers and 17.4 when we also extracted numbers from unfinished reasoning. So we choose to run all the analysis with finished reasoning as this seems to be cleaner, we will distinguish these inclusion rules. Restricting the other six models to the queries DeepSeek answered changed their pooled rate from {\color{#176537}\text{16.4\\% to 15.9\\%}}. Removing the same number of queries at random within each group also gave 15.9%. This does not bring down the robustness of the central claim.

You also raised the ambiguity in Figure 4. Using the original 34 estimators and five families, and sharing credit among near-ties, the pooled sinusoid result across six models retained the shift: from the two nearest bins to the two farthest, the Linear share fell 10 percentage points and the Periodic share rose 20. This strengthens the claimed change with query distance. Qwen2.5-14B and Gemma-3-4B retained both changes when we kept only unambiguous queries.

Q8. Why use GPT-OSS in Figure 2 if the argument concerns dense models?

As we mentioned in Section 4 GPT-OSS-20B is used as a fixed model not MoE, and the Figure 2 caption says “GPT-OSS fixed to 3.6B.”. We understand the confusion in naming and call this model GPT-OSS 20->3.6B. For this regression experiment, we fixed which experts which came active in a separate small regression task and used the same subnetwork for every query. Changing the query therefore could not switch the model to a different expert. We will make that detail explicit in the caption.

We also checked which numbers were plotted in Figure 2. Of the 400 completions, 352 reached a final answer. When we repeated the fit using only final answers, we still recovered two branches each of the four panels. We will revise Figure 2 to show completed answers.

Reviewer_GbuLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents in-context learning as hypothesis selection conditioned on queries. The examples within a prompt provide the model with several modes, and the specific input decides which one will be used. This is tested on simple tasks of predicting numbers (regression), with fixed demonstrations and only varying the input.

The authors report that the predictions split into multiple different branches, and this branching increases together with the distance from the demonstrations. Additionally, activation steering can move predictions in and out of a branch. The authors also find that the model's answer probabilities can shift between branches depending on the input, which cannot be explained by a fixed mixture. Small transformers trained from scratch rarely show branching, which suggests that branching depends on the training distribution.

Critical strengths
  • The evidence for branching has good controls. The authors provide a local test, which flags branching in almost 20% of windows far from the demonstrations, whereas controls with increasing noise or a flattening continuation are flagged in a small number of windows (ruling out a single predictor with increasing noise).

  • The main claim is examined through multiple independent angles. Besides the answer-probability analysis, the authors analyze the claim geometrically via the window test, causally (steering), and across demonstration draws (conditional-density test).

Critical weaknesses
  • There is little evidence that the branches are actual hypotheses. Concretely, the authors reported that "about half of flagged extrapolation windows contain two nearly constant levels," which suggests degenerate outputs. In the main steering example in Figure 3, 2/3 modes are constants, which indicates that the model may be failing (and not choosing a hypothesis). The answer-probability test is the most direct experiment. However, only 3/30 combinations pass it when the default settings are applied. The conclusion that the view "reconciles" Bayesian, kernel, and algorithmic accounts is not supported, nor is the claim that a little supervision at the query level is enough to steer modes in practical scenarios.

  • The steering evidence is missing an important control. To my understanding, the steering vector is here computed as the mean difference between activations for queries in one mode and queries in another one. If the queries in different modes also have systematic differences, the steering vector may partly encode the query itself. In this case, adding the steering vector could move predictions between modes without using any hypothesis. This could be settled by computing a vector between two groups of queries that have the same mode.

Questions for authors
  1. What happens if you compute the steering vector as the mean difference between two groups of queries within the same mode? Would this move predictions between the modes?

  2. How many discovered modes are constant, and do the answer-probability results still hold if tested only on non-constant hypotheses?

8 comments
AuthorsOpenReview ↗

A natural language task to assign semantic meaning to the hypothesis learnt in context

Thanks a lot for the deep and thoughtful reviews. To show that LLMs indeed learn meaningful hypothesys and switch between them based on queries, we use two new natural language 'like' datasets. Just like the regression task, LLM is given x->y pairs that could be explained by some rules and then asked to predict for a new x (x->?), where x and y have semantic meaning.

(1) Country task.\color{teal}{\text{Country task.}} We give pair of country and city for the LLM to learn from. In the demonstrations are pairs for which x is a county and y is both the capital was also the largest city. Both rules therefore fit every example. We then query 29 countries for which the rules (capital and largest) give different answers.

We asked the same 29 country questions with 10 different sets of demonstrations. This let us ask the two questions. Did the model hypothesis depend on the demonstration or query? We evaluate this on four different models and the results are:

ResultAcross the four models
Demonstration sets in which each model gave both kinds of answer20/20
Variance in capital-answer probability explained by query identity80.2–98.4%
Variance explained by the demonstration setAt most 1%
Answers naming the largest city40–105 of 580 per model

We compared countries by averaging across demonstration sets and compared sets by averaging across countries. The ranges cover four models, each tested with 10 and 20 examples per prompt. We computed these shares separately for each model and demonstration count. For the country effect, we averaged each country’s capital probability over 10 demonstration sets and compared the 29 country averages. For the demonstration-set effect, we averaged over the same 29 countries and compared the 10 set averages. Capital probability was normalized over the candidate city names for each query.

Capitals remained the dominant as one would expect. The largest-city/economic center was different from capital for all the query 29 countries. We then do steering and show that we can indeed show both the hypothesis at every query with an accuracy of 86% across the models. In short the depending on query LLMs used both hte hypothesis and they were steerable awith the same method as the paper.

Element task.\color{teal}{\text{Element task.}} We gave name of an element from the periodic table as x and the corresponding y for demonstrations satisfy (a)“the chemical symbol” and “the first letter of the name” (example Hydrogen-> H). On transductive querying for elements where the rule split, most models learnt rule (a) as expected 1,541 of 1,590 answers. But even in this setting a minority showed the rule (b). And this sample (80/20 split for constructing vector and evaluating) was enough to steer models into hypothesis (b) with an accuracy of 82%.

The evidence supports query-dependent expression of a rule in context.

AuthorsOpenReview ↗

Extrapolation windows containing constant levels vs degenerate outputs

Yes. We agree with you that LLM often give degenerate (yet a consistant mode) of prediction. For some of queries, it simply gives an output that is sometimes nowhere in the demonstration or looking like they belong to any meaningful hypothesis learned from the demonstrations. In the main text we often call this the collapse mode, but now following your comment we are calling this the degenerate mode. Sawtooth like functions where specifically used to elicit this mode more often.

Figure 3 shows the failure you describe. ** We gave Qwen2.5-7B 100 examples of the rising line y=2.5x−3y=2.5x-3, then asked it to predict yy for 500 values of xx. On 51 queries, it copied the first demonstration answer, −27.74, exactly. Another 228 predictions clustered around −273.45, far from any demonstrated answer. Only 43/500 predictions fell within 50 units of the true line. We will correct the caption and describe these plateaus as degenerate modes.

How often does this happen?\color{teal}{\text{How often does this happen?}} We went back to all 9,421 extrapolation windows across six models. Each window contains a small group of nearby queries outside the range of the demonstrated inputs. We now asked a new question: as xx changes, do both branches give different values of yy, or does one keep giving roughly the same number? We counted a branch as constant if its fitted value changed by less than 5%. This should not be read as degenerate for functions like sine as correct hypothesis would also read as constant, but we keep all function for completness. Also, these constant could be nearest neighbour not necessarily degenerate.

What did the model switch between?Flagged windows
Two nearly constant values1,025 (60.2%)
A nearly constant value and a sloped branch378 (22.2%)
Two sloped branches299 (17.6%)

This count tells us how often the model returns to a constant value. Then we took the constant windows and wrote hand crafted rules to avoid counting nearest neighbour or function mean output being read as degenerate. Then the number of degenerate windows went down from 82.4 % to 18.6%

To see if this effects the central claim, we add this to the null: could occasional fixed answers explain the branching?\color{teal}{\text{To see if this effects the central claim, we add this to the null: could occasional fixed answers explain the branching?}} We tested a predictor that follows a single smooth curve but occasionally gives a fixed number. We matched how often it did this to how often each model gave such answers. Then we looked specifically at windows where both fitted branches had a slope. For Qwen2.5-7B, our test flagged 145 of 556 such windows (26.1%), compared with 10.2% for the control showing the proposed hypothesis as the better explanation.

AuthorsOpenReview ↗

We agree to your critique on the claim and tone it down

What do our results say about theories of ICL?\color{teal}{\text{What do our results say about theories of ICL?}}

We asked the question: is ICL transductive, and gave evidence that, the model, given the same demonstrations follow different hypothesis under different queries. In the new country task, for example, every demonstration fits both rules. Yet the same model gives the capital for some countries and the largest city for others when queried. We did not test what the Bayesian, kernel and algorithmic accounts would each predict for those countries. We will therefore remove the claim that our results reconcile the three accounts from the abstract and conclusion. Thanks for pointing this out.

How much supervision is enough to steer a model?\color{teal}{\text{How much supervision is enough to steer a model?}}

In our steering experiment, we first identified the modes and then used examples from those modes to build a steering vector. The discovering of modes takes larger number of examples. To support the practical claim, we would need to show that a user with only a few labelled examples can reliably steer the model toward the rule they want on new queries. We did not establish that, and we have removed that claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.