acceptodds
Under review as a conference paper at ICLR 2027

Bifrost: Steering Strategic Trajectories to Bridge Contextual Gaps for Self-Improving Agents

Abstract

Autonomous agents excel in self-improvement through reflection and iterative refinement, which reuse successful task trajectories as in-context examples to assist subsequent reasoning. However, shifting across tasks often introduces a context mismatch. Hence, existing approaches either discard the trajectories or manipulate them using heuristics, leading to a non-negligible fine-tuning cost or unguaranteed performance. To bridge this gap, we reveal a context-trajectory correlation, where shifts in context are highly parallel with shifts in trajectory. Based on this finding, we propose BrIdge contextual gap FoR imprOvised trajectory STeering (Bifrost), a training-free method that leverages context differences to precisely guide the adaptation of previously solved trajectories towards the target task, mitigating the misalignment caused by context shifts. Our trajectory adaptation is conducted at the representation level using agent hidden states, ensuring trajectory transformation accurately aligns with the target context in a shared space. Empirical experiments reveal that Bifrost consistently surpasses existing trajectory reuse and fine-tuned self-improvement methods by 2%–14% across adaptation tasks in math problem solving (AQUA to GSM8K), question answering (ARC to GPQA), and code generation (HumanEval to LiveCodeBench), demonstrating that agents can effectively leverage past experiences despite substantial context shifts.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Autonomous Agents, Self-Improvement, Trajectory Adaptation, Representation Engineering, In-Context Learning, Training-Free Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tchbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Bifrost studies how to reuse successful LLM trajectories on tasks that differ from the source tasks. It keeps source trajectories in the prompt and adjusts a selected model layer using the difference between the target query’s representation and the average representation of the source trajectories. The model weights remain unchanged. The paper motivates this approach through linear representations and a Bayesian account of in-context learning. Experiments cover mathematical reasoning, scientific question answering, and code generation across three Llama model sizes. Further experiments examine transfer from mathematics to code, other model families, and the effects of steering direction, layer choice, demonstration count, and trajectory quality.

Critical strengths
  1. I like the simplicity of the method. It requires no weight updates, and the representations of source trajectories can be computed in advance. It also fits naturally into a system that already stores past solutions as demonstrations. If the gains are reproducible, this would be a useful way to improve the reuse of existing experience without adding another training stage.

  2. The reported gains extend across several tasks and model sizes. With the 8B model, Bifrost achieves 90.22% on GSM8K, compared with 87.41% for the strongest reported baseline. On LiveCodeBench, its pass@1 is 17.84%, compared with 14.98% for the strongest alternative. The experiments with other model families and transfer from mathematics to code add useful breadth. I also appreciate the controls in Table 8: changing the steering direction or randomizing the source trajectories reduces performance, providing some evidence that the particular intervention matters.

Critical weaknesses
  1. The theory does not yet justify the method as implemented. In Appendix D.1, the proof exchanges the order of JJ and Σ−1\Sigma^{-1}, then uses J⊤J^\top to recover a concept update from a change in hidden states. These steps require conditions that are not stated. The proof also treats the observed difference between the target query and source trajectory representations as a displacement toward an ideal target concept; I could not find a justification for this identification. Appendix D.2 then derives the bound by choosing an oracle coefficient, whereas the experiments use α=1\alpha=1. For a general coefficient, the bias includes an additional term, ∥P(Δ)∥2(α−α⋆)2\|P(\Delta)\|^2(\alpha-\alpha^\star)^2, which needs to be accounted for. Appendix E.2 provides a useful comparison between fixed and oracle coefficients, but this empirical check does not establish the stated bound. I would need these points resolved before accepting the theoretical explanation of why this particular steering rule should preserve useful source knowledge and improve adaptation.

  2. I found it difficult to judge how reliable the advantages over the strongest baselines are. The source trajectories are sampled randomly, but the main tables do not show how much the results vary across trajectory selections or generation seeds. This matters for gains such as the 1.01 percentage point improvement on GPQA-Diamond with the 1B model. Appendix F.2 reports three runs for the trajectory quality experiment, which is useful, but it does not resolve the uncertainty in the main comparisons. I also could not determine the full training settings for RISE and Paprika, or exactly what feedback the iterative methods receive when solving target tasks. The gains over ordinary ICL are encouraging. However, stronger claims about outperforming trained and iterative methods need clearer experimental details and evidence that the differences persist across runs.

Questions for authors
  1. What assumptions justify the mapping in Eqs. (15)-(17), and how does it apply to the empirical vector in Eq. (7)? Can you state the risk bound for the fixed coefficient used in the experiments, including any remaining error from the choice of coefficient?

  2. Appendix E.2 states that the tuning instances are disjoint from evaluation. Could you briefly explain how this separation was ensured for GPQA-main versus GPQA-Diamond and for the LiveCodeBench releases?

  3. How much source data was used to train RISE and Paprika, what optimization settings were used, and how were those settings chosen? What feedback did Reflexion and RISE receive during target evaluation, and how was that feedback obtained?

  4. Were Bifrost and ICL evaluated using the same source trajectories and decoding settings? Please report paired comparisons across independently sampled trajectory banks and generation seeds, with uncertainty estimates. Please also explain the difference between the 1B GPQA-Diamond results in Table 4, which reports 24.24%, and Table 9, which reports 44.44%.

1 comment

Thank you for the detailed response and the additional experiments. The pseudoinverse addresses my concern about the commutation step, and the four paired runs on Llama-3B strengthen the evidence for Bifrost’s gains over standard ICL. I remain uncertain about the stability of the smaller gain on GPQA-Diamond with Llama-1B and whether the gains over the strongest trained and iterative baselines hold across runs.

Reviewer_U2gZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Bifrost, a training-free method for reusing previously successful reasoning trajectories under context shifts. Bifrost computes a query-specific steering direction by subtracting the average hidden representation of source trajectories from the representation of the target query. It then adds this direction to selected hidden states while generating an answer conditioned on the source demonstrations and target query. The paper motivates this approach through a context–trajectory correlation and a Bayesian interpretation of in-context learning. Experiments cover mathematical reasoning, scientific question answering, and code generation, with reported improvements over prompting, trajectory-reuse, and parameter-updated baselines.

Critical strengths
  1. The paper address trajectory transfer problem with a novel and simple intervention. Historical trajectories can be hard to use in a new circumstance because of the contextual gap. Bifrost adds a vector to steer trajectories customized to the new context.
Critical weaknesses
  1. The paper did experiments only on Llama models. It would be good to see if this trick is useful across different model families. And it is not good to do experiments only on 2024 models nowadays. The authors should consider more advanced models.

  2. The improvement of Bifrost cannot be easily identified as a successful strategy transfer. Although the authors have done ablations, the method lacks sufficient interpretability. While the current experiments demonstrate that it is indeed effective, they do not clarify why these improvements were achieved.

Questions for authors
  1. Can the authors extend the experiments to other more advanced models? This is my main concern. And if this is completed, I will raise my score.

  2. What is the precise method for injecting the steering? Is the "last token hidden state" injected only once, which means at the final token of the prompt, or is it injected at every step of the autoregressive decoding process?

  3. Why do the GPQA-Diamond results differ between Table 4 and Table 9? Both models are named Llama-3.2-1B, but the Bifrost figures are 24.24% and 44.44%, respectively.

Reviewer_2npuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bifrost, a training-free method that uses hidden-state steering to adapt previous reasoning trajectories to new tasks. Experiments on math, QA, and code generation show promising improvements over existing baselines.

Critical strengths

1.The problem is interesting, and reusing previous trajectories without fine-tuning is a useful direction.

2.The method is simple and evaluated across multiple tasks and model sizes, with encouraging results.

Critical weaknesses
  1. I'm not fully convinced by the theoretical analysis. Theorem 1 does not directly establish the claimed context-trajectory correlation, and some derivations in Appendix D, particularly Eq. (17) and the use of Jensen's inequality, seem problematic.

  2. The experiments do not clearly isolate the benefits of trajectory reuse from general hidden-state steering. A stronger target-only steering baseline is missing. Also, the results are inconsistent: Table 4 reports 24.24% for Llama-1B on GPQA, while Table 9 reports 44.44%.

Minor Comments

1.Most experiments use small LLMs. Generalization to larger models remains unclear.

2.The correlation analysis reports p-values but lacks effect sizes.

3.Some figures are crowded and difficult to read.

Questions for authors

1.Could the authors clarify the theoretical derivations in Appendix D?

2.How does Bifrost compare against target-only activation steering?

3.Why are the GPQA results inconsistent between Tables 4 and 9?

2 comments
AuthorsOpenReview ↗

Author #1: Your Opinions about Our Theoretical Justification and Experimental Validation

Dear Reviewer 2npu,

We sincerely appreciate your help in providing constructive feedback. We earnestly value your comments and have carefully replied to every single point raised by you.

We have provided further theoretical insights justifying the interpretability of our contextual steering, and we have scaled our experiments to the Qwen series, which has demonstrated promising improvements. Moreover, additional validation and paper organization have further strengthened our contribution.

Could you please spare some time to check our rebuttal and confirm whether our efforts addressed your concerns? We are looking forward to hearing from you.

Best wishes,
Author #1.

Reviewer_2npuOpenReview ↗

Rebuttal Follow-up

Thanks for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, however, I'm still not fully convinced by the theoretical justification, especially the assumptions needed to connect the theoretical analysis to the actual steering method. While the additional experiments are helpful, I think the evidence for the proposed mechanism is still somewhat limited.

For these reasons, I'll keep my original rating for now. That said, I do see value in this work, and I'm open to acceptance if the other reviewers and AC feel the contributions are strong enough.

Comments on the paper

3 comments
AuthorsOpenReview ↗

[Multi-Author Discussion] We are here to help address further questions

Dear Reviewers,

Thank you for your support in providing valuable feedback.

We hope our rebuttal has adequately addressed the concerns raised. We remain open to further discussion and would be happy to provide additional clarifications or address any remaining questions.

In the rebuttal phase, we have multiple authors involved in this process (each will be numbered), and each one will provide insightful feedback from their own expertise, while focusing on the core insights of this paper. We hope that by introducing a multi-author discussion, we can provide comprehensive assistance to fully address the reviewers’ concerns. We thank you again for your help, and look forward to hearing from you soon.

Best,
All authors.

AuthorsOpenReview ↗

Author #3: Intuitive Explanation of Effectiveness

Dear Reviewers,

Thank you for your constructive feedback and continued discussion. I would like to provide three high-level insights into why Bifrost works, drawing on the Linear Representation Hypothesis (LRH) [1], while also addressing the recent questions regarding generalization to larger Qwen models and stability across runs.

1. Effective reasoning adaptation does not require knowing the ideal target concept.

Under LRH, related tasks can share transferable concept directions, while their contextual differences are reflected in hidden-state displacements. This suggests that the representation difference between previous trajectories and the target query can provide an informative direction for adapting reasoning.

Importantly, Bifrost does not require knowing the ideal target reasoning trajectory. Instead, it uses the target query to approximate a useful adaptation direction. Our controlled experiments support this intuition, showing positive alignment between contextual changes and corresponding reasoning-trajectory changes (cosine similarities of 0.34 and 0.39).

2. Bifrost adapts transferable reasoning concepts rather than merely steering toward the target query.

Previous trajectories contain reusable reasoning patterns, such as algebraic manipulation and logical deduction, while the target query specifies the requirements of the new task. Bifrost combines these complementary signals to adapt previously successful reasoning patterns to new contexts. This explains why Bifrost differs from generic activation steering. Our additional experiments show that Bifrost achieves 85.0% accuracy, outperforming target-only steering (75.5%), source-context suppression (82.0%), and random-source steering (79.0%).

This understanding also provides an intuition for generalization across model families and scales. Since Bifrost operates on semantic concept directions rather than model-specific parameters, its underlying mechanism can potentially transfer across models that exhibit similar linear concept structures. As model capacity increases, richer representations of transferable concepts may further facilitate effective adaptation, although improvement is not guaranteed to increase monotonically with scale. This intuition is supported by our new Qwen3.6-27B experiment, where Bifrost achieves 98.25% accuracy, outperforming the strongest baselines (97.00%) despite their already high performance.

3. Linear concept steering can produce stable improvements in complex, multi-step reasoning.

Although reasoning trajectories are complex and nonlinear, they are generated through successive token predictions. Under LRH, steering along meaningful concept directions can influence output logits, as characterized by Theorem 1. Since Bifrost applies steering throughout autoregressive generation, these changes can progressively shape the reasoning trajectory, as illustrated by the algebraic reasoning example in Fig. 2.

This mechanism also offers an intuition for stability across different runs and source trajectories. By averaging representations from previous trajectories, Bifrost can reduce the influence of individual source-specific variations while retaining shared transferable concepts. Moreover, its target-conditioned steering direction allows adaptation to depend on the conceptual relationship between source experiences and the target task, rather than relying exclusively on particular demonstrations.

Our additional evaluations support this interpretation: four paired runs on Llama-3B consistently demonstrate improvements over ICL, while further repeated experiments on Llama-1B GPQA-Diamond retain a positive mean improvement. Additional multi-run comparisons against RISE and DoT also show that Bifrost achieves the highest mean performance across the evaluated benchmarks.

Intuition Takeaway

Bifrost's effectiveness arises from three complementary mechanisms: previous trajectories provide transferable reasoning knowledge, the target query guides its contextual adaptation, and representation-level steering enables this knowledge to influence subsequent reasoning.

These mechanisms offer a unified intuition for why Bifrost can generalize across model families and demonstrate consistent improvements across experimental runs. More broadly, our findings suggest that self-improving agents can benefit not only from accumulating successful experiences, but also from effectively adapting and reusing those experiences under changing contexts, without expensive parameter updates.

We hope these perspectives further clarify the effectiveness and potential of Bifrost. We appreciate the reviewers' valuable feedback and welcome further discussion.

Kind regards,
Author #3

Reference

[1] Park et al., The Linear Representation Hypothesis and the Geometry of Large Language Models, ICML 2024.

AuthorsOpenReview ↗

Final Rebuttal Summary: Key Contributions, Addressed Concerns, and Reviewer Feedback

Dear Area Chair and Reviewers,

As the rebuttal discussion concludes, we sincerely thank the Area Chair and the reviewers for your time, constructive feedback, and thoughtful engagement. We briefly summarize the significance of our unique research problem, the contributions of Bifrost, our responses to the main concerns, and the positive developments during the rebuttal.


1. Research significance and contributions

Problem and significance. A fundamental goal of self-improving agents is to continuously accumulate and reuse past experiences to solve new problems. However, even previously successful reasoning trajectories can become ineffective or harmful when applied to tasks with different contexts. Existing approaches often discard such experiences or require costly retraining, leaving cross-context experience reuse as an important yet underexplored challenge for developing general-purpose, continuously improving agents.

Our solution. We propose Bifrost, a simple, training-free approach that enables agents to adapt past successful reasoning strategies to new tasks rather than directly reusing or discarding them. The key insight is that contextual differences between past and current tasks can guide how previous reasoning should be adapted. Bifrost captures these differences through model representations and uses them to guide reasoning toward the new task, effectively reusing past knowledge without additional training.

Generality and broader impact. Bifrost opens directions for post-self-improvement problems. It enables autonomous agents to leverage their experiences successfully under different contexts. For example, mathematical reasoning learned from previous tasks can be useful for solving physics problems; or an autonomous driving system operating in a new environment can adapt its existing knowledge instead of relearning from scratch. These findings open a promising direction toward more flexible and scalable self-improving agents that can transfer accumulated knowledge across diverse tasks, domains, and contexts.


2. Strengths recognized by reviewers

The reviewers recognized several strengths:

  • Novelty and significance (2npu, U2gZ): Addressing an important and underexplored problem of reusing past experiences across different contexts through a novel adaptation mechanism.

  • Simplicity and practicality (2npu, tchb, U2gZ): No additional training or parameter updates, making Bifrost efficient and readily applicable to existing self-improving agents.

  • Strong empirical effectiveness (2npu, tchb): Encouraging improvements across diverse tasks, model sizes, and architectures, supported by extensive comparisons and ablation studies.


3. Main concerns and rebuttal outcomes

We addressed the reviewers' major concerns through theoretical clarifications and additional experiments:

  • Theoretical justification (2npu, tchb): We strengthened the theoretical derivations, clarified the assumptions connecting latent concepts to empirical steering, and refined the risk bound to account for practical steering coefficients. Additional controlled experiments further support the proposed mechanism, with empirical steering achieving 0.80 cosine similarity.

  • Mechanism and interpretability (2npu, U2gZ): New controlled experiments demonstrate that Bifrost captures meaningful contextual and reasoning changes, while additional ablations show that its improvements arise from combining relevant prior trajectories with the target query, rather than generic activation steering.

  • Reliability and generalization (tchb, U2gZ, 2npu): We conducted additional multi-run evaluations, including comparisons with trained and iterative baselines, providing further evidence of consistent improvements. We also extended evaluations to newer and larger models, including Granite-4.2-3B, Qwen3-14B, and Qwen3.6-27B, where Bifrost outperforms these evaluated baselines.


4. Reviewer feedback and manuscript revision

We are pleased that two reviewers (U2gZ and tchb) have increased their ratings to 3 (weak acceptance) following our rebuttal and additional evidence. Reviewer 2npu explicitly acknowledges the value of our work and remains open to acceptance.

We have also revised our manuscript to incorporate the theoretical improvements, additional experimental evidence, and clarifications discussed during the rebuttal.


We believe the discussion strengthened the paper and addressed most actionable concerns. Taken together, the significance of the research problem, the generality and practical effectiveness of Bifrost, and the positive developments in reviewer feedback provide strong support for the contributions of this work. We hope this summary will be helpful in your final assessment.

Thank you again for your time and consideration.

Best regards,
Author #2 (on behalf of all authors)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.