- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates In-Context Learning (ICL) in large language models through controlled one-dimensional scalar regression tasks. Drawing on the human function-learning literature, the authors propose a view of ICL as query-conditioned hypothesis selection: demonstrations induce a repertoire of candidate predictive modes, while the query determines which mode is expressed. This perspective aims to extend accounts based on fixed implicit learning algorithms or demonstration-conditioned hypothesis mixtures.
The authors evaluate this hypothesis using eight open-weight autoregressive language models across ten synthetic data-generating functions, five demonstration densities, and five interpolation/extrapolation regimes. Their methodology combines several complementary analyses:
- Prediction-surface analysis: Sequential RANSAC peeling with simple function primitives (constants, lines, and low-degree polynomials) identifies branches in greedy prediction surfaces. Local interleaving tests and nonparametric modality tests across independently sampled demonstration sets provide additional evidence of predictive multiplicity.
- Activation steering: Interventions on intermediate residual-stream representations at the final input-token position test whether predictions on held-out queries can be redirected between identified branches.
- Pointwise attribution: Predictions are compared against a library of 34 classical estimators, grouped into five inductive-bias families (Linear, Polynomial, Tree, Kernel, and Periodic), under global and local fitting scopes.
- Output-probability analysis: Token-prefix expansion provides bounds on numerical-answer probabilities, enabling tests of whether fixed hypothesis-mixture weights can explain the observed query-dependent behavior under specified concentration assumptions.
- Controlled pretraining: Small GPT-2-like transformers trained on known linear and mixed function distributions are used to examine how pretraining support influences the emergence of multimodal predictive behavior.
Critical strengths
1. Complementary Methodological Approaches to Predictive Multiplicity
Rather than relying solely on aggregate prediction errors or visual inspection, the authors combine several complementary probes of ICL behavior: RANSAC-based surface decomposition, local branch-interleaving tests with matched single-function controls, nonparametric modality tests across demonstration sets, numerical-answer probability analysis, and causal activation interventions. In particular, the local branch tests and held-out steering experiments provide meaningful evidence beyond post-hoc curve fitting. This methodological breadth strengthens the paper's behavioral characterization of extrapolation in scalar-regression settings.
2. Conceptual Motivation and Transparent Reporting of Limitations
The connection to classical human function-learning research, including knowledge partitioning and hybrid rule–exemplar accounts, provides an interesting conceptual perspective on ICL. The authors also report several findings that qualify their central hypothesis, including substantial ambiguity in classical-predictor attribution (Table 11), sensitivity of probability-based conclusions to numerical acceptance intervals (Appendix J), and the limited emergence of multimodal behavior in transformers trained on controlled function distributions (Table 14). These analyses make the empirical scope and limitations of the proposed interpretation more transparent.
Critical weaknesses
- Internal Hypotheses vs. Numerical Decoding Artifacts While the local interleaving tests (Appendix I, Table 12) rule out simple continuous noise, interpreting prediction branches as distinct internal hypotheses remains unestablished:
- Decoding Artifacts: Autoregressive float generation introduces discrete effects (quantization, rounding, token-frequency bias) not captured by continuous baselines. Constant output branches (e.g., Figure 3) may simply reflect numerical token attractors rather than algorithmic hypothesis switching.
- Steering Specificity: Activation steering (Appendix E, Table 10) uses output-defined vectors with large perturbations (up to 4x mean difference) but lacks norm-matched random/orthogonal controls or magnitude-matched donor groups. This demonstrates token-level steerability rather than mediation of an abstract hypothesis-selection mechanism.
- Fragility, Sensitivity, and Narrow Scope of Reweighting Evidence The probability-based evidence for query-conditioned reweighting (Section 4.2, Appendix J) is highly brittle and conceptually restricted:
- High Sensitivity: Only 3 of 30 model–task pairs show certified reweighting and dominant switching at n=100. Expanding the acceptance band from 2% to 10% alters positive classifications from 2/60 to 17/60, showing heavy dependence on arbitrary thresholds.
- Restricted Mixture Test: The rejection applies only to candidates concentrating >=95% probability mass in designated bands (epsilon=0.05), not general fixed-weight Bayesian mixtures whose predictive modes naturally shift with queries.
- Attribution Ambiguity & Selection Bias: 61–65% of queries have ambiguous estimator attribution (Table 11). Furthermore, DeepSeek-R1 exhibits ~46–48% parse failures at n >= 80 (Table 2); dropping these introduces severe survival bias.
- Limited Generality: Controlled-pretraining tests (Appendix K, Table 14) rarely exhibit multimodal switching, indicating that the phenomenon is heavily contingent on specific 1D regression setups rather than a general property of ICL.
Questions for authors
1. Robustness to Numerical Representation and Decoding Controls
Do the recovered branches and local interleaving patterns persist when numerical formatting is modified, such as through changes in decimal precision, scientific notation, or rescaling outputs to integer-valued ranges? Can the authors evaluate the branch detector against controls that explicitly reproduce discrete numerical tokenization, quantization, or autoregressive decoding effects, rather than only continuous-noise perturbations?
2. Specificity of Activation-Steering Interventions
How do the reported induction and removal rates compare with those obtained using (a) random or orthogonal steering vectors of matched Euclidean norm and (b) donor groups matched for predicted output magnitude and query location? How large is the applied perturbation relative to the residual-stream norm at the selected layer? These controls would help establish whether the intervention specifically targets predictive hypotheses rather than more general output-correlated directions.
3. Robustness and Scope of Probability-Based Reweighting
Given that only 3 of 30 model–target-function combinations satisfy the joint reweighting and dominant-switching criterion at 100 demonstrations, how stable are these certificates under alternative candidate discovery sets and numerical acceptance-band definitions? Which classes of fixed-weight mixture models are excluded by the test, and which remain compatible with the observations under the stated 95% concentration assumption? Can the authors provide additional evidence that the conclusion is not primarily driven by the chosen candidate family or interval resolution?
4. Missing Predictions and Attribution Consistency
Can the authors report parsing success rates stratified jointly by model, target function, demonstration count, and extrapolation distance, and assess how excluding unparsed reasoning-model outputs affects the estimated distribution of predictive modes? Additionally, do the Linear-to-Periodic attribution trends in Figure 4 persist under tie-aware assignment using the same five-family taxonomy and estimator library as the main analysis, particularly given the high ambiguity rates reported in Table 11?
5 comments
2. Specificity of Activation-Steering Interventions
2. Does steering move the model from one rule to another?
We rebuilt the steering experiment with nearly the same input value, then compared it with 10 random and 10 orthogonal directions of the same length. The mode steering vector direction moved 167/210 predictions away from previous expressed mode. The strongest control baseline moved only 29/210. This answers the query-location concern for that example. It still matters where those predictions went. With a narrow band around each fitted mode, 84 of 101 departures landed outside every band for the control across the board.
| Direction | Removal at α = −1 | Induction at α = +1 |
|---|---|---|
| Original steering direction | 181/210 (86%) | 53/156 (34%) |
| Highest count among the 20 controls | 29/210 (13%) | 84/156 (53%) |
| Steering direction from 35 donor pairs matched on query location | 167/210 (80%) | 84/156 (53%) |
(b) Matched donors. After matching on query location, removal still exceeded every control. The donors’ absolute output ranges did not overlap (0.96–73.25 versus 227.12–325.46), so we could not also match on magnitude.
Perturbation size. At |α| = 1, the Figure 3 perturbation was 17.3% of the median residual-stream norm at the selected layer. Across the four regenerated Table 10 experiments, it was 4–21% at |α| = 1 and 16–84% at |α| = 4. The Figure 3 controls support suppression of one constant answer. Tests of transfer between two sloped branches gave no consistent effect.
3. Robustness and Scope of Probability-Based Reweighting
3. How much does the probability result depend on the candidate rules?
It depends on both the rules we test and how close an answer must be to count as a match. For each query, we measure how much answer probability falls near each candidate’s prediction. We then ask whether the same weight for that candidate could explain two different queries. This allows a rule’s prediction to change with the input.
| At 100 demonstrations | Qwen2.5-7B | Gemma-3-12B |
|---|---|---|
| Demonstration sets showing a weight change, using candidates found on separate discovery queries | 39/50 | 27/50 |
| Same test, using five candidates specified before seeing model outputs | 29/50 | 22/50 |
The full criterion also asks whether the highest-weight candidate changes. Three of 30 model–function combinations passed with discovered candidates. We repeated discovery on five fresh query grids. Under the new test, the full result held on 6/6 grids forall three Qwen sinusoid, Qwen piecewise-linear Gemma exponential, counting the original grid. Changing the band width also mattered. With the candidate list held fixed, all of the three combinations passed at all three widths tested (2%, 5% and 10%) on the new seed run we executed.
Your question also exposed a shortcoming the old test could call it dominant even if the remaining candidates together held more probability. We now require the winner’s lower bound to exceed the upper bound for every competing group. This removed the false switches in our fixed-weight simulations. The three combinations still pass at the original 5% width. We will restate the claim in the main text.
1. Robustness to Numerical Representation and Decoding Controls
Thanks a lot for your thoughtful reviews. This helped us improve the paper.
1. Do the branches survive when we change how numbers are written?
Thank you for the question. We do the exact robustness check you suggested. Yes, the proposed explanation hold the robustness checks.
The number format changes both the values the model repeats and how often it switches between them. We tested Qwen2.5-7B and Gemma-3-12B on the same linear and cubic functions, writing the numbers with two decimals, four decimals, as rescaled integers, or in scientific notation. Both models still produced repeated output levels in every format. There is some interesting changes in behaviour about which numbers they repeated. On the linear task, Qwen repeatedly gave answers between 213 and 337 with two decimals. Those large values disappeared with integers and scientific notation. The overall analysis suggest that the hypothesis learnt from the demonstrations depend on the format the demonstrations are presented, and the query conditionining is the dominant explanation.
To measure switching, we looked at 30 nearby queries at a time and counted a window when the predictions alternated between two fitted lines. On the Qwen linear task, 8 of 40 windows were flagged with two decimals, 7 of 40 when we repeated that same run, 3 of 40 with integers and 14 of 40 with scientific notation. So the format matters even when the underlying regression problem stays the same.
Could decoding alone produce these flags? We now add this to our null and started with a single smooth curve and altered its printed answers using each model’s own number tokens. We tried rounding, cutting the number at a token boundary and changing its first token. At the rates measured in the models, these controls flagged at most 13.5% of extrapolation windows, compared with 15.7% for the five models tested.
Your suggestion about repeated numbers gave a different result. We let the smooth curve occasionally return a fixed value, such as zero or the largest demonstrated label. At the pooled frequency observed across six models, this control flagged 21.1% of windows, compared with 18.1% for the models. A flag alone therefore gives us too little reason to infer a second rule. The clearest exception was Qwen2.5-7B. When both fitted lines had a slope, it flagged 145/556 windows (26.1%), against 10.2% for its control. It was the only model with enough such windows to show an excess. The overall takeaway remains that the hypothesis learnt from the demonstrations depend on the format the demonstrations, and across formats query conditioned switching holds. We will add the full robustness check analysis in the appendix.