- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SGG-ReflAct, a sub-goal-guided reflection framework for long-horizon LLM agents. The method first uses an LLM planner to decompose a task into a sequence of sub-goals. During execution, the agent dynamically identifies the active sub-goal at each step and jointly performs reflection and action selection based on the current environment state and the current sub-goal. The authors further introduce an enhanced variant, BeamSGG-ReflAct, which uses beam search to improve sub-goal planning and achieve better task decomposition. Experiments on three benchmarks, ALFWorld, ScienceWorld, and Jericho, demonstrate that SGG-ReflAct outperforms existing baselines in most setups, with particularly noticeable improvements for LLMs with fewer parameters. Further ablation studies suggest that the performance gains primarily come from explicitly anchoring reflection to the active sub-goal. The authors find that the proposed method reduces the invalid-action rate, while more sophisticated planning search does not necessarily yield further improvements, highlighting the importance of plan quality for long-horizon agent performance.
Critical strengths
-
The method is simple to implement, easy to adapt to different models and application scenarios, and introduces relatively low computational overhead. SGG-ReflAct is a runtime method that does not require additional model training or complex external modules. By simply combining sub-goal planning with sub-goal-anchored reflection, the method can effectively improve LLM performance on long-horizon agentic tasks. SGG-ReflAct only requires one additional LLM call for planning at the beginning of each task, without significantly increasing the per-turn runtime overhead. This makes the method highly practical and scalable.
-
The experimental evaluation is relatively comprehensive and provides evidence that the method can effectively improve runtime performance. The authors conduct experiments across multiple benchmarks and LLMs of different sizes. The results show that the proposed method improves task success rates in most settings, with particularly noticeable gains for smaller models. Also, the authors conduct ablation studies to analyze the respective contributions of sub-goal planning and sub-goal-anchored reflection. They also investigate the effects of invalid-action rates and planning quality on overall performance. These analyses help explain why the method works, rather than merely demonstrating performance improvements.
Critical weaknesses
-
The novelty of the proposed method is relatively limited, and its distinction from existing work is not sufficiently clear. The proposed method mainly combines sub-goal planning with step-wise reflection, replacing the final task goal with the active sub-goal as the target of reflection. However, similar ideas have already been explored in prior work. For example, Devil's Advocate (DEVIL’S ADVOCATE: Anticipatory Reflection for LLM Agents, Wang et al., 2024) also combines sub-task decomposition, pre-action reflection, and sub-task goal alignment to guide agent behavior. HiAgent (HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model, Hu et al., 2025) also investigates the use of sub-goals to organize long-horizon agent execution. Although the proposed method differs in its specific implementation, its overall technical novelty appears relatively limited. The authors should more clearly explain the key differences from these existing methods and provide more direct experimental comparisons to demonstrate the independent contribution of their approach.
-
The experimental design and ablation studies have several limitations that weaken the support for the paper's main claims. The key ReflAct+Plan ablation study is only conducted on ScienceWorld with Llama-3.1-8B-Instruct, making it unclear whether the findings generalize to other models and benchmarks. This ablation also does not clearly separate the contributions of active sub-goal identification and sub-goal-anchored reflection, leaving the source of the performance improvements insufficiently understood. The experiments use only a single random seed without a statistical reliability analysis, even though some of the reported improvements are relatively small. The paper also relies heavily on invalid-action rate as a proxy for environment grounding. This metric does not directly measure whether the agent correctly understands the environment state or identifies the appropriate sub-goal, and the results do not show consistent improvements across all models. These limitations make it difficult to fully validate the proposed explanation for the performance gains.
Questions for authors
-
How does SGG-ReflAct differ from existing methods listed in Critical Weaknesses section?
-
Can the authors further disentangle the contributions of active sub-goal identification and sub-goal-anchored reflection through a more controlled ablation study?
-
Can the authors provide stronger evidence of the statistical reliability of the reported improvements, given the use of a single random seed?
3 comments
W2. Scope of the ablation. This now covers seven settings (Table R1, General Response 2/5), including the two ALFWorld exceptions and their explanations.
Q2. Identification vs. anchoring. Because the identification and use of the active sub-goal occur within a single response, we cannot ablate one without the other. Identification is almost never wrong (regressions ≤ 0.68%, skips ≤ 1.63% of transitions in all nine settings; Table R6 in our response to rXbk). Since both ablation conditions utilize the same plan and the correct sub-goal is almost always identified, the difference comes from how the reflection uses that sub-goal. We will note the lack of a direct ablation as a limitation.
W2. The invalid-action rate is indirect. We agree and have added evidence categorized by failure types:
- False completion (Robotouille, GPT-4o): 17 of ReflAct's 32 failures involve declaring an unfinished dish complete; SGG-ReflAct has 1, and 9 of the 13 episodes solved exclusively by SGG-ReflAct are of this type.
- Procedural steps (runs of Table 1): on ALFWorld heat/clean/cool tasks, Llama-3.1-8B episodes performing the transformation increase from 12 to 40, and 19 of the 33 episodes solved exclusively by SGG-ReflAct were procedural failures under ReflAct. In ScienceWorld, failures involving repeated rejected commands drop from 52 to 33 (GPT-4o), accounting for 22 of the 30 episodes solved exclusively by SGG-ReflAct.
- Where it helps and where it does not (Table R4): the gain appears on tasks with more prerequisite steps when each sub-goal covers at most approximately two such steps, though the net gain remains bounded in part by low-level action loops (General Response 4/5). Anchoring can also persist in the case of an incorrect formulation (ScienceWorld "boil lead": 146 turns of rejected variants).
Q3. Reliability of single runs. Following ReflAct's single-run protocol, and given the cost of multi-seed re-runs during the rebuttal period, we have added paired exact McNemar tests (Table R5): four of the six ALFWorld/ScienceWorld settings show significant results; the gain is not significant for GPT-4o on ALFWorld, which is at the performance ceiling, or for GPT-4o-mini on ScienceWorld, and Jericho cannot statistically separate the methods. Additional runs on ALFWorld GPT-4o-mini yielded 74.6–79.9% (ReflAct+Plan, 4 runs) and 70.1–76.1% (SGG-ReflAct, 3 runs). We will report these tests alongside Table 1 and will not characterize non-significant settings as improvements.
Revisions: HiAgent and Devil's Advocate in Related Work; the seven-setting ablation (Table 14, in the revised PDF); identification rates and failure-type analyses in the appendix; and the lack of a direct ablation, the persistence of anchoring, and low-level action loops in the Limitations.
The authors' response has addressed my concerns, and I will raise my score to 3.
We thank the reviewer for their constructive review. Tables R1–R5 are provided in the General Response.
W1/Q1. Difference from HiAgent and Devil's Advocate. While all three methods utilize sub-goals, they differ in how they utilize them at each step. We have re-examined both papers in full:
The presence of sub-goals and their specific application are distinct issues. In HiAgent's own ablation, sub-goals without memory compression ("w. TD") reach 40.0% on Tyreworld, positioned between the standard agent (10.0%) and the full method (60.0%). Since porting either system to our environments would necessitate original design choices, we isolated our contribution through the controlled comparison in Table R1. We will add both papers to the Related Work section.