acceptodds
Under review as a conference paper at ICLR 2027

Learning Reusable Inference Across Representations

Abstract

Good in-context prediction does not establish whether a model has learned inference that transfers across representations. We construct a controlled test using latent parity tasks whose full-rank binary embeddings change representation reuse while preserving the Bayes problem and the statistical value of joint evidence. An exact hierarchy of support-label interactions quantifies this value: at latent rank , Bayes prediction needs supports and polynomial-time inference, whereas the optimal label-additive predictor needs supports at fixed risk. Across two contextual architectures, final-answer training exploits more of this opportunity with persistent embeddings, but that benefit transfers incompletely to unseen embeddings. Controls show that joint-label dependence is learnable when the organisation of the evidence is supplied; difficulty at higher ranks even with persistent embeddings motivates supervision of the inference process. A process-supervised recurrent learner executes autonomously under fresh embeddings, transfers beyond its training ranks, and turns additional supports into lower risk. These results separate statistical opportunity, representation reuse, and acquired computation, showing why evaluations of learned adaptation should test both what transfers and how effectively additional context is used.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
in-context learning, meta-learning, Bayesian inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jTy5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether a model gain inference capabilities across representations. This paper construct a latent parity task, where the embeddings can change but the latent geometries remain the same. Theoretically, this paper demonstrate that Bayesian prediction requires O(r)O(r) supports, while additive predictions requires Θ(2r)\Theta(2^r) to achieve a constant loss. Empirical results on looped ICL and Set ICL suggests persistent representations benefit but they cannot transfer to unseen ones completely. Additional elimination transitions trained on Set Transformer experiments are discussed.

Critical strengths
  1. This paper provides a well-controlled in context learning setup, where intervene the embedding does not change the latent prediction rule.
Critical weaknesses
  1. The hierarchy structure defined in Section 3.1 is insufficiently explored. The main comparisons regarding sample efficiency focus on Bayesian predictor and additive predictor lies in V1\mathcal{V}_1. However, Transformer models are much more expressive than V1\mathcal{V}_1 class since they can have higher order label interactions. Therefore, the theoretical results shows limited relevance to empirical results.

  2. The experiments section requires further clarification in the main text. For instance, Section 6 introduces elimination transitions and recurrent Set Transformers, but without clear description of new transitions, model architecture, etc. Further explanation on these essential components are necessary.

Questions for authors

Please refer to the weaknesses part.

  1. In Theorem 6, the sample complexity results are only about Bayes and V1\mathcal{V}_1. Is it possible to develop similar sample complexity results for general Vq\mathcal{V}_q?

  2. Could the authors further explain how theoretical results in Section 4 connected to provided empirical results?

4 comments
AuthorsOpenReview ↗

Interaction Hierarchy and Empirical Connection

We thank the reviewer for recognising the value of the controlled representation intervention and for the questions about the theoretical hierarchy and its empirical interpretation. We have revised Sections 3–6 to make these connections and the experimental methodology more explicit.

1. General interaction orders and sample complexity. Our theory is not restricted to additive predictors. Theorem 6 characterises optimal risk at every interaction order: R(Vq)=Pr⁡(kmin⁡>q)R(V_q)=\Pr(k_{\min}>q). Theorems 8–9 (Appendix D.4) further establish finite-length bounds and an interaction-order phase transition. In particular, for any fixed q≥1q\geq 1 and target risk ϵ∈(0,1)\epsilon\in(0,1), these bounds imply nϵ(Vq)=Θ(2r/q)n_\epsilon(V_q)=\Theta(2^{r/q}). We revised this and make the general-order consequence explicit in Section 4.2.

2. Connection between theory and experiments. We agree that Transformers can exploit higher-order label interactions. Our question is how much of this statistical opportunity they learn to exploit, and whether that capability transfers across representations. The interaction hierarchy provides an oracle benchmark, not a restriction on Transformer expressivity. V1V_1 is an optimal additive reference, not a restriction on the learned models. The gap R(V1)−R⋆R(V_1)-R^\star measures the total statistical benefit available beyond additive label dependence, and our empirical metric η\eta measures the fraction of this opportunity recovered by a trained model. Positive population-level recovery therefore certifies useful nonadditive label dependence, without claiming to identify the model's internal interaction order or inference algorithm.

The experiments further investigate what prevents expressive learners from recovering this statistical advantage. Section 5.1 shows that both contextual architectures can exploit joint-label evidence when its useful organisation is supplied, but struggle when they must discover the relevant relations themselves. At rank 6, revealing the exact route increases recovered opportunity from 0.002 under ordinary endpoint training to 0.998 (Appendix B.3, Figures 13–14). This implicates relation discovery as an important bottleneck, rather than an inherent inability to combine labels.

Additionally, Appendix B.3 (Figure 13) evaluates performance conditional on the minimum required interaction order kmin⁡k_{\min}. These results examine learned performance at different required interaction orders, rather than relying solely on the aggregate Bayes–additive comparison. Recovery also deteriorates with rank at fixed kmin⁡k_{\min}indicating that interaction order alone does not explain the empirical difficulty. Importantly, Proposition 4 establishes that the entire oracle interaction-risk hierarchy is unchanged across embeddings. The representation experiments test differences in learners' ability to exploit the same statistical opportunity, rather than differences in available information. We have clarified these connections in Sections 4–5.

3. Elimination transitions and architecture. We agree that Section 6 previously relied too heavily on the appendix. The revised section now describes the mutable state, pivot selection and XOR elimination updates, and the recurrent four-layer Transformer that predicts successive states. Training combines final-answer, state-transition and pivot supervision, with teacher forcing annealed away. Evaluation is fully autonomous, without teacher-provided transitions or routes. We also clarify the distinction between architecture-matched endpoint controls and the stronger curriculum comparison (full implementation in Appendix A.6).

4. Connection to real-data experiments. We have additionally evaluated two pretrained tabular ICL models across five real datasets (new Section 7, Table 2). These experiments extend the central question motivating our theory: whether learned inference remains effective when the representation changes but the predictive information available in context is preserved. Orthogonal feature rotations provide an information-preserving intervention, and we observe dataset-dependent changes in predictive performance and in the benefit obtained from additional support examples. Unlike latent parity these tasks do not admit the same tractable Bayes and interaction-order benchmarks. The new experiments test the broader relevance of representation sensitivity, rather than validating the specific interaction-order mechanism or measuring the recovery of its theoretical advantage.

We hope these revisions resolve the concerns about theoretical scope, empirical relevance and methodological clarity, and would welcome your reconsideration of the overall assessment in light of the revised manuscript.

Thank you again for your constructive review. As the discussion period draws to a close, we would welcome your thoughts on whether the revisions address your original concerns, and whether they warrant an updated assessment.

In particular, we have made the general interaction-order sample-complexity results explicit (Section 4.2), strengthened their connection to the empirical analyses of learned inference (Section 5), and substantially expanded the methodological explanation of recurrent elimination and its controls (Section 6).

We would be happy to clarify any remaining questions before the discussion closes.

Thank you for the additional results. My remaining concern is that the bounds for Vq\mathcal V_q give limited insight into how many support examples a trained Transformer needs to achieve a fixed risk as the latent rank grows. I will maintain my current score.

Thank you for clarifying your concern. We agree that deriving support-sample complexity bounds for trained Transformers would be an important next step towards a fuller learning-theoretic understanding of ICL.

Our work provides a foundation for that question by separating statistical opportunity from learned, reusable inference. Even when tasks require the same underlying computation and have identical oracle support requirements, our experiments show that endpoint-trained Transformers can exploit the available evidence under one representation without reliably transferring that ability to another. Process supervision improves this transfer, demonstrating that the way inference is learned matters alongside the number of supports available.

We believe establishing this distinction is a necessary step towards explaining trained-model sample efficiency: before characterising how many examples a model needs, we must distinguish the information those examples provide from the inference capability the model has actually acquired.

We hope this clarifies how our results contribute to a broader learning-theoretic understanding of ICL.

Reviewer_288WOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

In-context learning is about a language model being fed patterns as a prompt and inferring how to continue the pattern. This is done without retraining or finetuning the model, the instructions/patterns/examples are provided in the prompt to the language model, and from those the language models have been shown to be able to infer this pattern. There have been works exploring if this inference procedure generalizes across different samples of one task as well as even across different tasks. However this paper investigates a third generalization axis, whether the model might memorize the representation of the task or generalize across that.

The task chosen by the paper is a latent parity task, which is a chained XOR problem where only a subset of the input bits participate in the XOR. This subset is chosen according to a hidden rule. And the inference procedure is given input-output examples, also called support set, of this chained XOR problem and then given a new input should predict what the output should be. However importantly the actual inputs are transformed with a full-rank binary embedding matrix G. This is to explore the representation generalization of the inference procedure.

They train two transformers, Looped-ICL and Set-ICL. And evaluate them with the fraction of Bayes–additive opportunity recovered. They have two theoretical baselines which are the Bayes predictor and the best label-additative predictor. In the first experiments they see that these transformers do learn ICL but that the representation of the task matters, and they do not generalize for it. The author subsequently introduces extra supervision in the training which aims to mimic the Gaussian elimination steps (as that is an effective solution to the latent parity task). This is shown to improve the representation generalisation, however limited to lower rank problems still.

Critical strengths

Novelty. Whilst it is common to investigate ICL generalization in regards to the samples of a task as well as across different tasks, the paper explores a novel research question regarding the generalization capabilities in regards representation of this task.

Theoretical foundation. The paper has a solid theoretical basis, with well chosen theoretical baselines. The propositions are well motivated and they have good control experiments. It is well suited for the chosen experimental setup, however if this setting adequately addresses the research question is open for discussion (See Relevance beyond latent parity tasks).

Critical weaknesses

Presentation and clarity. The paper is unnecessarily complicated to follow, it is written in dense telegraphic language, making the motivation, research question, methodology, experiments difficult to follow. The paper often introduces terminology (which to the best of my knowledge is not common in the field) without adequate explanations. It would greatly benefit from a rewrite as in its current form the paper is very hard to understand, especially for the broader ICLR community.

Relevance beyond latent parity tasks. The experimental results are restricted to only the latent parity task and thus it is not clear how well the findings translate to harder more realistic tasks. However I do understand the need for a controlled setting to evaluate the research question.

Limited generalization. The process supervision does indeed improve the generalization performance, however the performance still deteriorates greatly at higher ranks for the latent parity tasks. Thus even with this relatively simple task, the experiments do not conclusively show an inference procedure that is capable of generalizing to harder problems under different representations.

Questions for authors
  1. Do you expect these results to translate beyond latent parity tasks? Are there any experimental observations suggesting the same for more realistic ICL tasks?

  2. Even with the process supervision the performance is decreasing greatly with higher ranks, what part of the method do you think is responsible for this?

  3. What is the reasoning behind the choice of a Looped ICL and Set ICL?

3 comments
AuthorsOpenReview ↗

Clarity, Generalisation and Methodology

We thank the reviewer for recognising the novelty and theoretical foundation of our work, and for their detailed criticisms. We agree that the original presentation was overly compressed in parts and that the empirical scope and limitations needed clearer explanation. We have substantially revised the exposition and added real-data experiments.

Presentation and clarity. We have revised the Introduction and Sections 3–6 to explain the central ideas before formalising them. In particular, Section 3 now introduces the interaction hierarchy intuitively, Section 4 uses a concrete XOR example to explain the representation intervention, and Section 6 describes the recurrent model's state, elimination transitions, supervision and autonomous evaluation directly in the main text. We also clarify the experimental controls and distinguish representation transfer from extrapolation to higher latent ranks.

Q1. Relevance beyond latent parity. We conducted additional experiments with two pretrained tabular ICL models (TabICLv2 and TabPFN-3.5-Fast) on five real OpenML datasets. Tabular ICL closely matches our labelled support-set/query formulation. Applying orthogonal feature rotations to supports and queries preserves predictive information and Euclidean geometry, yet at 2,048 supports, predictive log loss increased by approximately 28–31% on Spambase and 33–71% on Mozilla4. A Euclidean 5-NN control remained invariant. These effects are dataset-dependent, and rotations can also change the observed benefit from additional context. These results establish representation-sensitive prediction in real world ICL models, providing evidence of broader relevance without attributing the effects to the latent-parity mechanism. (new Section 7, Table 2; Appendix A.11).

Q2. Higher-rank generalisation. We agree that process supervision does not establish complete algorithmic generalisation. Our narrower finding is partial transfer to held-out ranks 6–7 after training on ranks 3–5, together with improved utilisation of additional supports and recurrent depth (Figure 5). This tests a non-trivial extrapolation to unseen ranks where the model must execute its learned (reusable) inference procedure on problems of greater intrinsic complexity than those encountered during training, rather than merely generalising to new examples at trained ranks.

However, the performance deteriorates at higher ranks, where the learned procedure might have to coordinate more elimination steps, creating opportunities for errors in pivot selection and state updates to accumulate. Appendix B.2 reports imperfect transition accuracy and rollout survival, consistent with this explanation, although the relative contributions of these failure modes have not been causally isolated. We have clarified these limitations in Section 6.

Importantly, our central finding does not require complete rank-general algorithm learning. The controlled representation experiments establish that successful inference within a persistent representation need not transfer to unseen representations, despite unchanged statistical opportunity. Process supervision is a complementary investigation of whether more reusable computation can be acquired, with encouraging but limited success.

Q3. Architecture choices. Looped ICL and Set ICL provide complementary contextual architectures: the former iteratively updates a query/workspace against support memory, whereas the latter jointly processes supports and queries. Testing both helps establish whether the observations are specific to one architecture, without claiming exhaustive coverage. The process-supervised learner uses a separate recurrent architecture, and the revised Section 6 distinguishes architecture-matched endpoint controls from the stronger curriculum comparison (details in Appendices A.3 and A.6).

We hope these revisions address the concerns about clarity, external relevance and empirical interpretation, and would welcome reconsideration of the overall assessment in light of the revised manuscript.

Presentation and clarity. The paper has indeed improved on its clarity, especially in the Introduction. However section 3-6 would benefit from more work on the structure and the formulation of some paragraphs.

Q1. Relevance beyond latent parity. The experiements added are interesting, although it does show that the predictive information is indeed preserved during representation transfer. Except for the larger support sets.

Q2. Higher-rank generalisation. The paper does indeed not count complete rank-general algorithm learning as a finding, and does not say that the process supervision is the solution to representation transfer. However the claims in the paper are made weaker by the these limitations of the process supervision as well as the fact that in more relevant tasks the representation transfer seem to work adequatly (atleast up to a certain size of the support set).

That being said, I will increase my score as the presentation/clarity has improved and additional experiments evaluating more relevant tasks have been added.

Thank you for revisiting the manuscript and for recognising the improvements in clarity and the added real-data evidence. We appreciate your remaining concerns regarding higher-rank generalisation and the limits of the process-supervision results.

We agree that some of the real-world datasets and support sizes show relatively little degradation, while others show substantial changes despite preserved predictive information. This reinforces our central question, which goes beyond predictive robustness - 'to what extent have models learned inference capabilities that can be reused across representations, rather than relying on familiar feature structure?'

Our controlled study investigates this by holding the statistical benefit of combining evidence fixed and examining whether learned models continue to exploit it when representations change. The real-data results illustrate why this distinction is relevant beyond the controlled setting.

Thank you again for your constructive feedback and for your updated assessment of the revised manuscript.

Reviewer_ZrbSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that good ICL capability does not imply that ICL transfers under representation shift. In order to study this systematically in a controlled setting, the paper proposes a latent parity task, where representations can change without undermining the information contained in the data. Through experiments, the paper shows that tested learners can combine labels when given useful combinations.

Critical strengths
  • Rigor: the paper provides extensive proofs to show that their synthetic latent parity task is the right task to study.
  • Presentation: although I think that a couple of diagrams to illustrate the tasks would have helped, I think the presentation of the arguments and the problem the paper is trying to tackle is good. The figures were also easy to understand.
Critical weaknesses
  • Significance: Although I understand that good ICL capability may not imply good transferability, it is unclear to me whether this is a bottleneck or an important problem in recent AI models. It would be very beneficial for the authors to show real-data examples that support the importance of the problem they are addressing.
Questions for authors
  • Could the authors explain the implications of their studies on ICL with real (e.g., natural language) data? 
  • The paper lacks the motivation behind why we should be interested in this setup. Could the authors show examples where real-world applications fail due to the lack of transferability of ICL capability despite high ICL performance?
5 comments
AuthorsOpenReview ↗

Significance and Real-Data Evidence

We thank the reviewer for recognising the rigour of our controlled study and for highlighting the need to establish its significance for contemporary ICL models. We agree that this connection needed to be clearer, and have strengthened the motivation and added experiments on real data.

Real-data evidence (significance and Q1). We evaluated two pretrained tabular ICL models (TabICLv2 and TabPFN-3.5-Fast) on five real OpenML classification datasets. Tabular ICL provides a natural first setting for this investigation because its labelled support-set and query formulation directly matches our theoretical setup. Applying the same orthogonal feature rotation to supports and queries preserves predictive information and Euclidean geometry. Nevertheless, at 2,048 supports, predictive log loss increased by approximately 28–31% on Spambase and 33–71% on Mozilla4, while a Euclidean 5-NN control remained invariant. The effects vary across datasets, and changes in predictive performance do not necessarily correspond to reduced benefits from additional examples. These results demonstrate that representation sensitivity is relevant to pretrained ICL models on real prediction tasks (new Section 7, Table 2; full results in Appendix A.11).

Implications for NLP (Q1). The same underlying question is relevant to language-model ICL: when a model learns to infer a rule from examples, can it reuse that ability when the task is presented differently? For example, a model may successfully combine evidence from demonstrations in one prompt format, but struggle to do so when the same task is expressed using different templates, label conventions, or wording. We expect this distinction to be particularly important for NLP tasks where combining evidence across examples/tokens offers a substantial advantage over treating them separately. Our results therefore motivate evaluating whether learned inference transfers across such changes, rather than measuring performance only on familiar prompt formats. We do not directly test language models, where ensuring that changes in presentation preserve the available information is more difficult. Our tabular experiments provide a controlled demonstration of representation sensitivity in pretrained ICL models and a starting point for studying this question in NLP.

Why this matters (Q2). Strong performance on new tasks does not necessarily establish that the underlying inference capability transfers to unfamiliar representations. This distinction matters for evaluating ICL models across datasets and feature pipelines, and ultimately for developing methods that can exploit information from new examples more reliably under representation changes. Our real-data experiments demonstrate an observable form of this sensitivity, rather than relying solely on hypothetical failures.

Role of the controlled study. Our latent-parity framework provides the complementary theoretical contribution. It preserves Bayes risk and the entire support-label interaction-risk hierarchy across representations, allowing us to quantify the statistical benefit of combining evidence and measure whether learned models exploit and transfer that benefit. The real-data experiments establish the broader relevance of representation sensitivity, while the controlled setting allows us to investigate its relationship to efficient inference precisely.

Manuscript revisions and scope. We have made this distinction explicit in the Introduction, added the five-dataset study and context-size comparisons in Section 7, and clarified the implications for evaluating reusable inference in the Discussion. These are controlled interventions on real prediction tasks, not documented deployment failures or evidence that the same internal mechanism explains both settings.

We hope these additions address your central significance concern and would welcome reconsideration of your overall assessment in light of the revised evidence and motivation.

Thank you again for your constructive review. As the discussion period draws to a close, we would welcome your thoughts on whether the revisions address your original concerns about broader significance, and whether they warrant an updated assessment.

We have strengthened the motivation and added experiments with pretrained ICL models on five real datasets (Section 7). These reveal heterogeneous sensitivity to information-preserving representation changes, including differences in how models benefit from additional context. This reinforces our central question: to what extent have models learned inference capabilities that can be reused across representations? Our controlled study provides a theoretical foundation for investigating this question by quantifying the statistical benefit of combining evidence and examining whether learners acquire and transfer that benefit.

We would be happy to clarify any remaining questions before the discussion closes.

Thank you for the detailed response and the additional experiments with pretrained tabular ICL models. However, my main concern about the significance of the findings remains unresolved. The new results demonstrate representation sensitivity, but do not establish that it is a major bottleneck. As you acknowledge, the experiments involve controlled interventions rather than documented deployment failures and the effects vary across datasets. I will therefore maintain my current rating. Evidence that limited transfer constrains models in practice would help me reconsider my rating, for example if language-model ICL performance degrades when the same task is expressed using different templates, label conventions, or wording. Alternatively, a concrete proposal for revising existing approaches in light of these findings, tested even in a small-scale experiment, would also give me reason to reconsider.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank the reviewers for their constructive assessments. In response, we have revised the manuscript to strengthen its motivation, clarify the theoretical and empirical connections, and establish broader relevance through new experiments on pretrained ICL models.

The central contribution is a controlled study of reusable inference in ICL. A central goal of ICL is to learn how to infer from examples in a way that can be reused across new tasks and representations, rather than relying on the particular form in which a task is presented. Yet strong ICL performance does not necessarily mean that the learned inference procedure will transfer to unfamiliar representations. Understanding this distinction is important for developing models that can efficiently adapt to new problems.

We develop a controlled latent-parity setting where combining evidence across support examples offers an exponential advantage in sample efficiency: Bayes prediction requires O(r)O(r) supports, while the optimal label-additive predictor requires Θ(2r)\Theta(2^r). Crucially, changing the representation preserves this advantage, along with Bayes risk and the entire support-label interaction-risk hierarchy. This makes the setting a useful test bed: the benefit of joint inference is substantial and precisely measurable, and we can study whether models learn to exploit and transfer that benefit without changing the information available to them. Our experiments show that persistent representations often do not transfer to unseen representations, while process supervision improves the use of additional context and enables partial generalisation to unseen task ranks/increased task difficulty. New experiments with pretrained tabular ICL models further demonstrate that representation sensitivity also arises on real datasets.

Broader implication. Strong ICL performance on new tasks does not by itself establish that a model has learned a broadly reusable inference procedure. Our findings motivate evaluating not only predictive performance, but also how effectively models use additional context and whether that capability transfers across representations.

Our theory quantifies the statistical benefit of combining support-label evidence and establishes its invariance across representations. Our learned-model experiments examine whether models exploit and transfer this benefit, while our new real-data experiments show that representation sensitivity also arises in pretrained ICL models. We distinguish what inference is statistically possible from what models learn and how well that capability transfers, providing a foundation for studying more generally reusable inference procedures.

The principal revisions are:

  1. Real-data evidence (Section 7, Table 2; Appendix A.11): New experiments with two pretrained tabular ICL models across five real datasets demonstrate representation-sensitive prediction under information-preserving feature rotations. On Spambase and Mozilla4, rotations increase predictive log loss by approximately 28–71%, depending on the model and dataset. We also show that changes in predictive performance and in the benefit from additional context need not coincide.

  2. Theory–experiment connection (Sections 3–5): We clarify the general support-label interaction hierarchy and its sample-complexity consequences beyond additive prediction. We also explain how the empirical benchmarks measure the recovery of the statistical advantage offered by joint evidence, without assuming a particular internal inference mechanism.

  3. Presentation and methodology (Sections 3–6): We simplify the conceptual explanations, make the recurrent elimination procedure explicit in the main text, and clarify architectural choices, experimental controls and the limits of higher-rank generalisation.

The Discussion has also been revised to bring together these complementary findings and clarify their implications and limitations. We address each reviewer's specific questions in the corresponding discussion threads.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.