acceptodds
Under review as a conference paper at ICLR 2027

The Goal Gets in the Way: Subgoal Relay for Long-Horizon LLM Agents

Abstract

Language model agents often decompose long-horizon tasks into subtasks and use the final goal to organize and guide execution. Our motivation experiments reveal a contrast: using the final goal to clarify a subtask improves local completion, yet providing the goal again with the same explanation reduces completion. Direct goal guidance may lead the executor to reconsider earlier planning decisions, causing unproductive actions and rework. We therefore propose Subgoal Relay (SR) to limit the final goal's direct influence on local decisions while retaining its guidance for the current subtask. We aggregate goal information using attention weights from the current subgoal to the final goal. We fuse this information into the subgoal value cache and bound the update magnitude relative to the original values, forming a relay for local execution. The model draws on this relay and the retained context for its decisions, while attention access constraints block direct access to the original final-goal field. We evaluate SR on ALFWorld, ScienceWorld, and WebShop across four agent frameworks and six model backbones, measuring task performance and end-to-end execution cost. In the main experiments with Qwen3-8B under a shared attention backend, SR increases ALFWorld success rates by up to 7.8 percentage points and mean scores on ScienceWorld and WebShop by up to 4.6 and 8.7 points, respectively, while reducing mean end-to-end execution time by 2.5–11.1%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Long-Horizon Tasks, Goal Interference, Hierarchical Goal Control, Attention Masking

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_p7SmOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

In long-horizon tasks, a widely adopted approach is the hierarchical one, where the overall goal is divided into several sub-goals and then executed. The authors observe that the overall goal, when applied to the execution of sub-goals, has both a positive effect of providing supplementary information and a negative effect of disrupting the actual execution. This paper introduces Subgoal Relay, a training-free intervention that transfers overall goal information into the current subgoal’s value cache using subgoal-to-goal attention weights, bounds the update magnitude, and masks direct response attention to the original goal field. Experiments across three environments, four agent frameworks, and six backbones show consistent improvements in mean task performance, with reduced end-to-end execution time in the main experiments.

Critical strengths
  1. The paper is well-written and easy to follow. The overview of Subgoal Relay is clear.
  2. The paper distinguishes the overall goal's role in local executing, framing a novel problem and intervention.
  3. The experiments cover multiple benchmarks and backbones, providing solid evidence.
Critical weaknesses
  1. The paper lacks discussions about the conditions where direct overll goal access becomes harmful.
  2. The experiments do not directly establish what information is transferred to the subgoal values or whether it specifically captures requirements relevant to the current subtask. A targeted representation analysis or counterfactual relay intervention would strengthen the explanation.
Questions for authors

See Critical Weaknesses above.

Reviewer_U1oXOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Subgoal Relay (SR), a novel technique designed to improve how language model agents execute complex, long-horizon tasks. The authors identify a counterintuitive problem in existing agent frameworks: while a final goal is necessary for high-level planning, continuously exposing the agent to this final goal during local subtask execution causes interference. The agent tends to second-guess its earlier planning decisions, leading to unnecessary rework, unproductive actions, and lower completion rates. To solve this, SR acts as an information filter. It aggregates necessary context from the final goal using attention weights, fuses this limited information into the current subgoal value cache, and then blocks direct attention access to the original final-goal text during local decision-making. By giving the agent only the relayed context it needs without the distraction of the overarching objective, SR significantly improves task success rates across multiple benchmarks while simultaneously reducing end-to-end execution time.

Critical strengths
  • The paper effectively highlights and addresses goal interference, a phenomenon where providing too much high-level goal context actively harms an agent's ability to complete low-level subtasks.
  • The proposed method manipulates attention weights and the value cache at inference time. By caching the relayed information and blocking direct access to the goal field, it solves the problem without requiring computationally expensive model retraining or fine-tuning.
  • SR improves success score and reduces mean end-to-end execution time rather than having a trade-off between them.
Critical weaknesses
  • Because SR relies on explicit KV cache manipulation and custom attention masking during decoding, it cannot be used with proprietary models. It is strictly limited to open-weight models where the user controls the inference engine.
  • The relative update budget ρ\rho controls how much the goal values alter the subgoal cache. While showing the ablation studies, this is an additional hyperparameter that must be tuned on a validation set. It is unclear if ρ\rho needs to be drastically retuned for entirely new domains or massive scale shifts in model size.
Questions for authors
  • How does SR compare to simply using negative prompting instructions? For example, adding a system prompt like 'Do not attempt to solve the final goal or reconsider previous steps until this subgoal is complete.' Does SR fundamentally outperform instruction-based focus?
  • You tested missing information and conflicting subgoals. However, what happens if the high-level planner hallucinates a completely incorrect subgoal? Does SR trap the agent in a failure loop faster than the baseline method?
Reviewer_96VVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether continued exposure to the final goal can disrupt local subgoal execution. It proposes Subgoal Relay (SR): use subgoal-to-goal attention to transfer goal information into the subgoal’s value cache, bound the update, and block direct response attention to the original goal. Higher-level planning retains goal access. This requires no additional training. Validation spans three benchmarks, four agent frameworks, six backbones, and five paired evaluations, supplemented by component ablations, behavioral diagnostics, and execution-cost measurements.

Critical strengths
  1. All 12 main Qwen3-8B framework–benchmark comparisons have positive paired 95% intervals for performance gains. ReCAP improves by 7.8 percentage points on ALFWorld and 4.6/8.7 score points on ScienceWorld/WebShop. Comparisons share model weights, controllers, and budgets; configuration selection uses separate development data. This is the strongest acceptance argument: SR provides a useful improvement across substantially different agent configurations.

  2. Meaningful evidence supporting the proposed mechanism. The paper examines masking, transfer, textual handoff, and transfer direction separately. With update magnitudes controlled, subgoal-conditioned transfer achieves 73.9% local completion versus 68.4% for uniform transfer. Additional diagnostics show recovery of goal requirements lost through masking, supported by human-reviewed annotations. These controls make the explanation credible beyond aggregate benchmark gains.

Critical weaknesses
  1. The distinctive relay’s added task-performance benefit is not consistently established. Compared with Mask-Only, SR’s gains have intervals crossing zero on ALFWorld, +4.0 [−0.3, 8.8], and ScienceWorld, +2.9 [−0.4, 6.4]. Comparisons with textual handoff are similarly inconclusive there. WebShop provides positive evidence, but superiority over simpler alternatives remains uncertain on two benchmarks. This limits how strongly the results justify the specific cache-transfer design.
  2. Deployment-level efficiency remains unvalidated. End-to-end comparisons use SDPA math for both methods; the optimized FlashAttention-2 baseline appears only in a fixed-workload microbenchmark. The paper’s own sensitivity analysis shows that an 8.3% reduction in baseline model time would erase the ScienceWorld speed advantage. Although transparently acknowledged, this qualification materially weakens efficiency as an acceptance argument.
Questions for authors
  1. What instruction-generation prompt, output budget, and regeneration/reuse policy were used for textual handoff? These choices determine whether its substantial runtime penalty represents a competitive implementation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.