acceptodds
Under review as a conference paper at ICLR 2027

Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMs

Abstract

Large language models (LLMs), when acting as agents, are expected to take observed data in context, infer the latent state space underlying the world, and leverage it for downstream prediction. However, prior work demonstrated that LLMs struggle to use representations learned in context on a graph tracking task, where the model needs to construct a representation of the graph governing data generation process and use it for subsequent predictions. In this paper, we show that extending this to few-shot settings, where each demonstration is generated from a different world with either the same or different graph topologies, enhances its prediction on 6 models from 4 model families. To understand this improvement, we linearly probe a low-dimensional world representation that encodes graph information in the hidden states. Notably, we find that few-shot demonstrations relocate the world representation and increase its predictive use. Specifically, for each model, these world representations shift in directions nearly orthogonal to their original subspace, and interventions on these representations selectively impair performance more than interventions on other subspaces. Consistent with this insight, we show that few-shot demonstrations with observations from different worlds improve performance on ARC-AGI-1&2, web agent tasks, and Othello. Our findings elucidate the role and internal mechanisms of few-shot demonstrations in in-context world modeling. More broadly, our work advances our understanding of how LLM agents learn from in-context observations and provides implications for their further improvement.

Primary area:
interpretability and explainable AI
Keywords:
In-Context Learning, Large Language Models, Few-Shot Learning, Mechanistic Interpretability, Representation Geometry, World Representations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_sQRvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether few-shot demonstrations help LLMs use in-context world representations. Across several graph tasks, demonstrations improve accuracy without substantially changing probe performance, but shift the representation subspaces. The authors use activation interventions to study this effect and report additional results on ARC-AGI, Mind2Web, and Othello.

Critical strengths
  1. The paper addresses an interesting question about the difference between constructing a world representation and actually using it for prediction.

  2. The improvement in accuracy despite largely unchanged probe performance is an interesting finding, especially since it appears across several models and graph topologies.

  3. The authors provide a fairly thorough analysis, including cross-shot probing, subspace comparisons, and activation interventions.

Critical weaknesses
  1. The accuracy gains may reflect a simple elimination strategy rather than better use of world representations. In the 4×4 lattice and triangle tasks, this strategy could achieve roughly 33% accuracy, close to the reported results.

  2. The causal evidence remains limited. Most Qwen3 intervention effects at 16 shots are not significantly positive (Fig. 7), and the increase appears largely driven by changes in the random control (Fig. 25).

  3. The practical-task results lack simple baselines, such as input copying for ARC-AGI (cell-wise accuracy) and random empty-square prediction for Othello, making the gains difficult to interpret.

Questions for authors
  1. Can the authors report accuracy among the three candidates remaining after elimination in the 4×4 lattice and triangle tasks, as well as how often the model predicts one of these candidates? This would help rule out a simpler explanation of the few-shot gains.

  2. Could the authors report the absolute effects of graph and random interventions separately, and test whether their difference changes significantly from zero to 16 shots? This would help clarify the interpretation of Fig. 7.

  3. Have the authors tested which parts of the demonstrations matter? For example, would instruction–answer pairs without random walks, or demonstrations with shuffled answers, produce similar gains? Simple baselines for ARC-AGI and Othello would also help interpret the practical results.

5 comments

Reviewer sQRv,

We have given your feedback careful consideration and greatly appreciate its contribution to our work.

Candidate Elimination (W1&Q1&Q3)

The accuracy gains may reflect a simple elimination strategy rather than better use of world representations

Thank you for this constructive feedback. Candidate elimination explains part of the few-shot gains, but our additional experiments show that it does not fully account for the improvements, especially in larger models.

We conducted additional experiments on 1) prompting with shuffled random walks in open Qwen3 models, 2) evaluating larger closed models (GPT6.1 Terra, Gemini-3.8 Flash, Qwen3.7 Plus), and 3) measuring conditional accuracy after candidate elimination.

As shown in Fig. 17 (App. D.2), small Qwen3 models achieve accuracy nearly comparable to the original prompt even when the random walks are shuffled. This indicates that the accuracy gains from few-shot demonstrations are confounded by candidate elimination, rather than arising solely from the predictive use of representations. However, Fig. 19 (App. D.4) shows that the conditional accuracy (the fraction of correct predictions among samples in which the model predicts one of the three candidates in the 4×4 lattice and triangle grid) exceeds the random baseline and improves as the number of shots increases. These results isolate performance after accounting for candidate elimination.

Interestingly, when evaluated on larger closed models, the original prompting substantially outperforms prompting with shuffled random walks, as illustrated in Fig. 18 (App. D.3) (eg, GPT6.1 Terra exceeds 80% with 32 shots). Furthermore, as shown in Fig. 19 (App. D.4), the conditional accuracy after candidate elimination also increases substantially with the number of shots. Thus, performance gains that cannot be explained by the candidate elimination shortcut are clearly observed at larger model scales.

Although candidate elimination partially accounts for the few-shot gains, we still observe performance improvements without it. We identify predictive use as one contributing factor, and our claims regarding representation analysis and random interventions still hold. We discuss this in the paper.

Predictive Use of Representation (W2&Q2)

The causal evidence remains limited test whether their difference changes significantly from zero to 16 shots?

First of all, our intervention is noisy due to estimation errors in the rank-rr subspace identified by SVD of the linear probe weights WW trained on 500 samples, and the difficulty of removing graph information from a low-dimensional subspace. We therefore compare graph interventions against random interventions in an orthogonal subspace, matched in L2 norm to the graph component of the current centered representation at each layer and node. We compare against random interventions because we intervene only at token positions corresponding to the random walk in the query, while zero-shot and few-shot settings differ in context, potential for repair, intervention magnitude, and baseline correct answer logits, making direct comparisons unreliable.

Following the reviewer's comment, we conducted a statistical significance test to verify whether the difference between graph and random intervention effects increases from 0-shot to 16-shot, and have added this analysis to Tab.1 (App.D.8). This shows that 12 out of 15 (3 Qwen3 models ×\times 5 topologies) remained statistically significant (p<0.05).

The trend is consistent across three Qwen3 models, Llama-3.2-3B, Ministral-3-8B-Base, and Gemma4-12B models (App.E.1), different rank-rr (App.E.2), and different random seeds (App.E.3). Please see below for further support.

Additional Experiments

We measured 1) normalized DE of instruction (task mappings) in Fig.8 and Fig.30 (App.D.7), and 2) cross probe AUROC of a linear probe trained on random walks and evaluated on instructions in Fig.29 (App.D.7).

As shown in Figs. 8 and 30, while the normalized DE of random walks remains unchanged across different numbers of shots (as in Sec.4.1), the normalized DE of instructions decreases as the number of shots increases. In contrast to the zero-shot case in Sec.4 of [1], this indicates that although few-shot demonstrations do not alter how well observations (random walks) are aligned with the latent graph topology, they enable the model to better map the fruits vocabulary onto the topology within the instructions.

Furthermore, when a linear probe trained on random walks is evaluated on instructions (Fig. 29), providing few-shot demonstrations yields a higher AUROC compared to the zero-shot setting. This means that the underlying graph structure can be read out more effectively from the instruction nodes.

Baselines for Practical Tasks (W3&Q3)

The practical-task results lack simple baselines

We added the baselines for ARC-AGI (input copying and color 0 padded) and Othello (uniform sampling from 64 squares and empty square) in Fig.9.

Other Prompting Baselines (Q3)

instruction–answer pairs without random walks

We conducted additional experiments on other prompting baselines: 1) Demo Walk Only: Includes only random walks in demonstrations (omits instruction, 2) Demo Instruction Only: Includes only instructions in demonstrations (omits random walks). 3) Demo Instruction with Random Mappings: Retains demonstration random walks, but replaces target answers with uniformly sampled random nodes in Fig.17 (App.D.2). All three baselines underperform the original few-shot prompting. This highlights the importance of demonstrations that pair random walks with task instruction (task mappings).


References

[1] Lepori et al. Language Models Struggle to Use Representations Learned In-Context. ACL2026.

AuthorsOpenReview ↗

Gentle Reminder

Reviewer sQRv,

Once again, we would like to express our gratitude for your critical review.

We (i) evaluated candidate elimination, including additional closed-source models (W1, Q1, Q3), and confirmed improvements on other tasks (Appendices D.2, D.3, & D.4), (ii) clarified the predictive use of representations (W2, Q2) and further supported our claim with normalized DE on instructions and attention analysis (Sec. 5.1), (iii) added a baseline for practical tasks (W3, Q3; Fig. 8), and (iv) added other prompting baselines (W3).

We would be grateful if you could review our responses and revisions when you have a chance. Thank you very much for your time and consideration.

Reviewer_rZ6vOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether few-shot demonstrations can help low-level models (LLMs) make subsequent predictions using world representations built from contextual observations. In a two-step right-shift graph prediction task, zero-shot hints already include random walks and task mapping examples, while few-shot hints add complete demonstrations from other worlds. Cross-world demonstrations improve accuracy across five topologies, even though graph information is already decodeable under zero-shot conditions, and adjacency detection AUROC and normalized Dirichlet energy remain largely unchanged. Changes in the decodeable subspace and hidden state centroids support the authors' view that demonstrations improve the reading and use of existing representations. Matching ablation experiments provide some evidence for improved prediction usage, but do not directly confirm that representation relocation is the cause of the improved reading rate. Other experiments show improved performance on ARC-AGI-1/2, Mind2Web, and Othello tasks, although the same representation mechanism is not validated in these tasks.

Critical strengths

1. Investigating Interesting Capability Boundaries Through Controlled Comparison. Under all experimental conditions, the final query remained constant, while the demonstration experiments altered the assignment of lexical items to nodes. This design aimed to examine how a complete demonstration from another world could improve predictions of the current world, thereby reducing the likelihood of directly replicating its answers. Multiple models and topologies in Figure 3, and demonstrations of different topologies in Figure 15, extend the empirical findings.

2. Extending Evidence Beyond Probe Decodability Through Meaningful Interventions and Sensitivity Analysis. The authors combined adjacency probing, Dirichlet energy, cross-experimental decoding, and subspace analysis with interventions in graph-related directions along the query path. Randomized control was applied to the ordering and perturbation norm of each node in each layer. This provides stronger evidence for the predictive use of world representations compared to simply using PCA visualization or a decrease in probe accuracy.

Critical weaknesses

1. The causal relationship between representation relocation and improved readout capability remains unclear. Lens vectors capture average displacements, while translations commonly found in node differences used in adjacency detection cancel each other out. Existing ablation experiments support the predictive role of graph-related directions but do not directly examine whether relocation itself improves zero-shot prediction capability. Subspace variations also lack references for independently trained detectors under the same lens condition. Cross-lens intervention comparisons rely on the normalization of clean probabilities for each condition, and some confidence intervals contain zero values. Direct representation exchange and more robust effect measurements would strengthen the proposed causal explanation.

2. Explanations and practical guidance for effective representation use are limited. An interesting finding is that graph information already exists in the zero-shot hidden state, and cross-world demonstrations may enhance its predictive capability. Although this paper changes the number of demonstrations and topology, it does not clarify which components of the complete demonstration are crucial. This also fails to explain why zero-shot models cannot effectively utilize this information, thus limiting guidance for designing more effective or concise cues. Identifying these factors will deepen the understanding of the mechanism and provide more operational guidance for practical cues.

Questions for authors
  1. When retaining the task mapping example but replacing the query traversal with an unordered fruit vocabulary, or shuffling the vocabulary order while keeping length and frequency constant, how much accuracy can be maintained? These control variables help distinguish between dependencies on graph structure and strategies such as candidate word elimination.

  2. Can you explore two-step relationships to differentiate between improvements in task-related information construction and increased use of existing representations? Similar adjacency relationships alone cannot fully distinguish these interpretations using AUROC.

  3. Given the improvements demonstrated by different topologies in Figure 15, do their effectiveness still depend on the task structure of the shared query? How do partial demonstrations (observation only or task mapping) or demonstrations of different prediction tasks affect performance?

  4. How do your findings and proposed representation relocation mechanism relate to existing theories of contextual learning?

7 comments

Reviewer rZ6v,

We sincerely appreciate your insightful feedback, which we have seriously considered to improve our work.

Predictive Use of Representation (W1&Q2)

Interpretation of Intervention Experiments:

The causal link between representation relocation and improved readout remains unclear

Our intervention experiments test the causal contribution of graph representations by ablating the principal subspace from which the graph is linearly decodable. We interpret the change in representation geometry as subspace relocation and use the Shot Vector to characterize this shift. However, we do not claim that the Shot Vector itself explains the performance gains.

To compare zero-shot and few-shot settings, we measure the effect of graph-subspace ablation relative to matched random interventions, since baseline performance and answer token logits differ between the settings. The larger relative effect in the few-shot setting suggests that the model makes greater predictive use of graph representations.

We view this increased predictive use as one factor contributing to the few-shot performance gains, while acknowledging that other mechanisms may also contribute.

We did additional experiments below to support our claims.

Additional Experiments:

distinguish improved construction of task-relevant information from increased use of existing representations?

We measured 1) normalized DE of instruction (task mappings) in Fig.8 and Fig.30 (App.D.7), and 2) cross probe AUROC of a linear probe trained on random walks and evaluated on instructions in Fig.29 (App.D.7).

As shown in Figs. 8 and 30, while the normalized DE of random walks remains unchanged across different numbers of shots (as in Sec.4.1), the normalized DE of instructions decreases as the number of shots increases. In contrast to the zero-shot case in Sec.4 of [1], this indicates that although few-shot demonstrations do not alter how well observations (random walks) are aligned with the latent graph topology, they enable the model to better map the fruits vocabulary onto the topology within the instructions.

Furthermore, when a linear probe trained on random walks is evaluated on instructions (Fig. 29), providing few-shot demonstrations yields a higher AUROC compared to the zero-shot setting. This means that the underlying graph structure can be read out more effectively from the instruction nodes.

Together with our intervention results, we believe these findings provide further evidence supporting the predictive use of internal representations.

Practical Implications (W2)

It also leaves open why zero-shot models fail to use this information effectively, limiting guidance on designing more effective or concise prompts. Identifying these factors would deepen the mechanistic insight and provide more actionable guidance for practical prompting.

Failure cases of zero-shot models have already been discussed in [1], which instead shows that graph tracking performance improves when grid coordinates are explicitly provided (Sec. 4 in [1]) or hints about the graph topology are included in the prompt (App. J in [1]). However, embedding the underlying states (topology) of observations (random walks) directly into the prompt is practically infeasible. So, in our work, we demonstrate that providing few-shot demonstrations (observations + task instructions + answers) yields modest performance gains. This suggests that agents interacting with an environment can improve their performance in-context. We believe this offers practical implications for existing endeavors that utilizes ICL for RL [2] and self-distillation [3].

Candidate Elimination (Q1)

distinguish dependence on graph structure from strategies such as candidate elimination.

Following this constructive feedback, we did additional experiments on 1) prompting with shuffled random walks in open Qwen3 models, 2) evaluating larger closed models (GPT6.1 Terra, Gemini-3.8 Flash, Qwen3.7 Plus), and 3) measuring conditional accuracy after candidate elimination.

As shown in Fig. 17 (App. D.2), small Qwen3 models achieve accuracy nearly comparable to the original prompt even when the random walks are shuffled. This indicates that the accuracy gains from few-shot demonstrations are confounded by candidate elimination, rather than arising solely from the predictive use of representations. However, Fig. 19 (App. D.4) shows that the conditional accuracy (the fraction of correct predictions among samples in which the model predicts one of the three candidates in the 4×4 lattice and triangle grid) exceeds the random baseline and improves as the number of shots increases. These results isolate performance after accounting for candidate elimination.

Interestingly, when evaluated on larger closed models, the original prompting substantially outperforms prompting with shuffled random walks, as illustrated in Fig. 18 (App. D.3) (eg, GPT6.1 Terra exceeds 80% with 32 shots). Furthermore, as shown in Fig. 19 (App. D.4), the conditional accuracy after candidate elimination also increases substantially with the number of shots. Thus, performance gains that cannot be explained by the candidate elimination shortcut are clearly observed at larger model scales.

Although candidate elimination partially accounts for the few-shot gains, we still observe performance improvements without it. We identify predictive use as one contributing factor, and our claims regarding representation analysis and random interventions still hold. We discuss this in the paper.

Other Prompting Baselines (Q3)

How do partial demonstrations (observations or task mappings only) or demonstrations of a different prediction task affect performance?

We did additional experiments on other prompting baselines: 1) Demo Walk Only: Includes only random walks in demonstrations (omits instruction, 2) Demo Instruction Only: Includes only instructions in demonstrations (omits random walks). 3) Demo Instruction with Random Mappings: Retains demonstration random walks, but replaces target answers with uniformly sampled random nodes in Fig.17 (App.D.2). All three baselines underperform the original few-shot prompting. This highlights the importance of demonstrations that pair random walks with task instruction (task mappings).

Connections to Theoretical ICL Literature (Q4)

How do your findings and the proposed representation-relocation mechanism relate to existing theories of in-context learning?

At a high level, our graph tracking task involves inferring a latent world representation zz from in-context observations DD and using it to predict yy given a query xx. This relates to Bayesian posterior prediction, p(y∣x,D)=∫p(y∣x,z)p(z∣D)dzp(y\mid x,D)=\int p(y\mid x,z)p(z\mid D)dz, used to explain ICL [4]. [5] show that explicitly inferring latent variables does not necessarily improve OOD generalization due to ineffective downstream use, closely relating to our distinction between representation construction vs. predictive use. [6] then show that amortized point estimators often outperform amortized posterior estimators. While our work does not compare these estimators, we empirically show that few-shot demonstrations from different worlds promote the predictive use of the target world's representation in LLMs.


References

[1] Lepori et al. Language Models Struggle to Use Representations Learned In-Context. ACL2026.

[2] Hübotter et al. Reinforcement Learning via Self-Distillation. ICML2026.

[3] Shenfeld et al. Self-Distillation Enables Continual Learning. ICML2026.

[4] Xie et al. An Explanation of In-context Learning as Implicit Bayesian Inference. ICLR2022.

[5] Mittal et al. Does learning the right latent variables necessarily improve in-context learning? ICML2025.

[6] Mittal et al. In-Context Parametric Inference: Point or Distribution Estimators? 2025.

Reviewer_rZ6vOpenReview ↗

Response to Authors' Rebuttal

Thank you for the detailed reply and extensive new experiments. On W2, if graph structure is already decodable from random-walk representations (Figure 4), what prevents its effective use for prediction? As a diagnostic, could you compare attention from the answer-prediction position to the final query’s random-walk versus instruction tokens in zero-shot and few-shot settings.

Reviewer_U4MgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how few-shot demonstrations affect the construction and predictive use of in-context world representations in LLMs. The results suggest that few-shot demonstrations relocate the probed world representations without substantially improving their decodability.

Critical strengths

The paper studies an interesting problem concerning the gap between constructing world representations and using them for downstream predictions. The combination of linear probing, representation analysis, and causal interventions provides useful empirical insights.

Critical weaknesses
  1. Each demonstration contains a random walk sequence, task instruction examples, and a correct answer. Increasing the number of demonstrations also increases the context length and changes the position of the final query. Therefore, could the performance improvements in Figure 3 be attributed to the model learning input-output mappings, rather than more effectively utilizing the inferred world representations?

  2. Figure 5 shows that few-shot demonstrations change the subspace in which world representations are encoded, while Figure 7 suggests that the model relies more heavily on these representations for prediction under few-shot prompting. However, these two observations do not directly establish that the subspace changes are responsible for the performance improvements.

  3. Section 5.2 demonstrates performance improvements from few-shot demonstrations on ARC-AGI, Mind2Web, and Othello. However, the authors do not investigate whether similar changes in world representations occur in these tasks.

Questions for authors

Refer to Weaknesses

2 comments

Reviewer U4Mg,

We truly appreciate your valuable and insightful feedback, which improves our work.

Few-Shot Performance (W1)

Therefore, could the performance improvements in Figure 3 be attributed to the model learning input-output mappings, rather than more effectively utilizing the inferred world representations?

First, we would like to clarify that each few-shot demonstration, in our settings, are from different worlds (ie, different fruits are assigned to the node). So, the demonstrations do not leak information about the input-output mappings in the query task.

Following the reviewer's suggestion, we conducted additional experiments on different prompting baselines to disentangle the effects of random walks and instructions (input–output mappings). We tested 1) Demo Walk Only: Includes only random walks in demonstrations (omits instruction, 2) Demo Instruction Only: Includes only instructions in demonstrations (omits random walks). 3) Demo Instruction with Random Mappings: Retains demonstration random walks, but replaces target answers with uniformly sampled random nodes in Fig.17 (App.D.2). All three baselines underperform the original few-shot prompting. This highlights the importance of demonstrations that pair random walks with task instruction (task mappings).

Predictive Use of Representation (W2)

However, these two observations do not directly establish that the subspace changes are responsible for the performance improvements.

We would like to clarify this point.

Our contribution is to show that few-shot demonstrations change the principal subspace from which representations are decoded. We also find that interventions on these representations have a greater effect in the few-shot setting than in the zero-shot setting, relative to random interventions (Fig.7).

As noted in the Limitations section , the model's use of these representations for prediction is only one possible factor behind the performance improvement. We describe the change in the representation subspace using the Shot Vector, which captures the direction of this shift (Fig.5(c)). The Shot Vector points in a direction distinct from those spanning the subspace, providing evidence that the subspace itself has shifted. We do not claim that the Shot Vector alone accounts for the performance improvement.

Our contribution builds on the finding in [1] that LLMs struggle to use representations in the zero-shot setting. We examine the corresponding few-shot setting, where demonstrations consist of observations, instructions, and answers.

Please let us know if you have any further concerns.

Representation Analysis of Practical Tasks (W3)

However, the authors do not investigate whether similar changes in world representations occur in these tasks.

It would be nice to investigate whether similar changes in representations occur in practical tasks. However, we found it difficult to conduct comparable representation analyses on these tasks because the ground-truth world states needed for probing are not readily available.

In ARC-AGI, the underlying world representation largely corresponds to "rules", and to the best of our knowledge, there is no established method for probing these rules from hidden states. In web agent tasks, the underlying application state and logic behind the observed HTML are not accessible.

In Othello, linear probes can classify each of the 64 squares into three states (eg, MINE, YOURS, or EMPTY) [2, 3]. However, the complexity of the state space makes it difficult to determine whether the geometry of the probe subspace reflects the overall world representation. Prior work also suggests that these linear directions may be just part of a richer, compositional representation [4].

For these reasons, in our main results, we introduced a graph tracking task following [1], where the world states and transitions are explicitly defined. Extending this analysis to more complex practical tasks remains an important direction for future work.


References

[1] Lepori et al. Language Models Struggle to Use Representations Learned In-Context. ACL2026.

[2] Li et al. Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task. ICLR2023.

[3] Nanda et al. Emergent Linear Representations in World Models of Self-Supervised Sequence Models. 2023.

[4] Lee et al. Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions. 2026.

AuthorsOpenReview ↗

Gentle Reminder

Reviewer U4Mg,

Once again, we sincerely thank you for your thoughtful reviews

In particular, we (i) added more baselines to address the concern that few-shot performance may simply result from learning input-output mappings (W1), (ii) clarified the predictive use of representations (W2) and further supported our claim with normalized DE on instructions and attention analysis (Sec. 5.1), and (iii) explained why we did not (cannot) probe representations in practical tasks (W3).

We would appreciate it if you could take another look at our responses and revisions when you have a chance. Thank you very much for your time and consideration.

Comments on the paper

2 comments

Dear Reviewers,

We thank all reviewers for their helpful feedback. Below, we summarize the revisions made to the paper and the additional experiments.

Additional Experiments

  • Appendix D.2: Other baselines for few-shot promptings.
  • Appendix D.3: Few-shot performances on 3 larger closed models (GPT6.1 Terra, Gemini3.8 Flash, Qwen3.7 Plus)
  • Appendix D.4: Evaluations excluding candidate eliminations across 7 models
  • Section 5.1 & Appendix D.7: Normalized DE of Instructions across different number of shots, and cross probe accuracy of a linear probe trained on random walks and evaluated on instructions
  • Appendix D.8: Hypothesis Testing on Graph and Random Interventions across zero-shot and few-shot setttings

Revisions

  • Reference to the additional experiments above in the main part.
  • Baselines for practical tasks (Fig. 9).
  • Limitation and future works paragraph.

In response to the reviewers' feedback, we have clarified the following points: 1) candidate elimination partly explains the performance gains in graph tracking, but performance still improves with more shots even without it (Appendix D.4). This improvement also grows with model size (Appendices D.3 & D.4). 2) To further support our intervention results, we conducted statistical significance tests (Appendix D.8) and measured normalized DE for instructions. As the number of shots increased, normalized DE decreased for instructions but remained stable for random walks (Section 5.1 & Appendix D.7). Overall, we believe these results support our hypothesis that few-shot demonstrations affect the predictive use of world representations. We have also clarified that this mechanism is not the only possible explanation for the performance gains, added baselines for practical tasks, and expanded the limitations.

Additions and changes are highlighted in blue in the PDF.

All in all, we sincerely appreciate the reviewers' critical comments and contributions to this work.

AuthorsOpenReview ↗

Additional Experiments Following Reviewer Feedback

Following reviewer rZ6v's insightful suggestion, we also conducted an attention analysis and added the following:

Section 5.1 (Fig. 10) & Appendix D.9 (Fig. 34): A comparison of the shares of attention weights and attention-weighted value vector norms assigned to random walk tokens.

The results show that few-shot demonstrations shift attention toward the random walk, from which we probe the representations, relative to the instructions.

Again, we thank all reviewers for their valuable feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.