acceptodds
Under review as a conference paper at ICLR 2027

Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Progress Verification

Abstract

Robotic foundation models have advanced manipulation, yet long-horizon tasks remain challenging under partial observability and uncertain execution. Reliable task completion requires retaining relevant information, verifying intermediate outcomes, and adapting plans to execution feedback. We introduce Goal2Skill, a dual-system framework that couples adaptive planning, proactive memory, and a self-evolving verifier. A high-level VLM constructs and revises sub-task plans, while a low-level VLA executes action chunks. Periodic verification connects these processes by using current observations and task memory to determine when to continue, advance, or trigger reflective correction. Memory preserves information needed by upcoming steps and records previous attempts, supporting targeted revisions while retaining valid progress. The verifier combines lightweight models, rule-based checks, and VLM reasoning, and refines its assessment procedures through execution feedback, with candidate updates validated before adoption. Experiments on RoboMME and RMBench show that Goal2Skill achieves the highest average success rate among non-oracle methods, with particularly strong gains on tasks requiring persistent memory and multi-stage coordination.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Long-horizon Manipulation, Adaptive Planning, Embodied System

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YRtfOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Goal2Skill combines a high-level vision-language planner, a low-level vision-language-action policy, task memory, and a verifier that decides whether to continue, advance, or replan. Experiments on RoboMME and RMBench report strong results on memory-dependent tasks, including 60% success on Blocks Ranking Try.

Critical strengths
  1. The system uses execution evidence to revise the remaining plan while retaining verified progress.
  2. Its memory and verification components target concrete challenges in multi-stage manipulation.
Critical weaknesses
  1. Several figures have a conspicuously AI-generated appearance, with inconsistent or distorted visual details that detract from their clarity and the overall professionalism of the presentation.
  2. The related-work discussion does not sufficiently position Goal2Skill against closely related dual-system and memory-centric frameworks, including RoboCerebra and RoboMemArena, making it difficult to assess the novelty of the architecture and the incremental contribution of its memory and verification components.
  3. Evaluation is concentrated on RoboMME and RMBench, without a real-robot study in this paper.
  4. The verifier depends on task-specific checks and may misjudge progress when observations are ambiguous.
Questions for authors

Please see the weaknesses section.

3 comments
AuthorsOpenReview ↗

Response to Reviewer YRtf

Thank you for your constructive comments and for recognizing the value of evidence-guided plan revision and our memory and verification mechanisms. We clarify the presentation, methodological distinctions, evaluation scope, and handling of ambiguous observations below.

W1: Figure quality and clarity.

Thank you for pointing out these presentation issues. The figures are created by the authors using Adobe Illustrator and Matplotlib without using AI tools. We will review the figures for inconsistent details, unclear annotations, and formatting issues, and improve their readability and visual consistency. We would also appreciate pointers to particular figures or elements that appear distorted so that we can address them precisely.

W2: Differences from RoboCerebra and RoboMemArena.

Thank you for highlighting these closely related works. We haved added explicit citations and comparisons in the revised manuscript.

The hierarchical framework introduced in RoboCerebra uses its high-level VLM to assess execution progress and update subgoals or refine instructions. PrediMem, introduced with RoboMemArena, predicts subtasks from recent observations and keyframe memory; its asynchronous inference procedure updates the active subtask whenever a new high-level prediction arrives, without describing a separate completion-verification gate.

Goal2Skill shares the use of high-level reasoning and memory with these frameworks. Our contribution lies in how verification and memory govern task progression and recovery. First, verification uses explicit checking procedures that are functionally distinct from subtask generation and are refined through execution feedback and replay validation. Periodic verification links frequent action execution with sparse planning: confirmed completion advances the existing plan, while detected failure or an exhausted subtask budget triggers reflection and targeted correction. Second, memory updates are conditioned on the remaining plan, preserving information needed by future steps alongside verified outcomes and previous recovery attempts. This context supports subsequent planning, verification, and recovery.

To assess the contribution of these design choices within Goal2Skill, we added two controlled comparisons.

1. Verification procedure versus planner-assessed completion. We replace our verification procedure with completion judgments produced by the high-level model that also generates subtasks. These judgments directly control task progression. We evaluate this alternative on 50 episodes per task across all 16 RoboMME tasks.

TaskFull G2SPlanner-assessed completion
BinFill58%2%
PickXtimes92%26%
SwingXtimes56%4%
StopCube2%0%
VideoUnmask84%64%
VideoUnmaskSwap24%30%
ButtonUnmask70%0%
ButtonUnmaskSwap50%6%
PickHighlight76%8%
VideoRepick60%22%
VideoPlaceButton52%26%
VideoPlaceOrder90%66%
MoveCube68%42%
InsertPeg2%2%
PatternLock32%14%
RouteStick8%6%
Average51.5%19.9%

Full Goal2Skill improves average success by 31.6 percentage points. This supports the contribution of our verification procedure as a whole relative to the tested planner-assessed alternative.

2. Structured memory versus recent execution history. Since both related frameworks already use memory, we examine whether our memory configuration provides benefits beyond access to recent execution records. We replace structured memory with the last eight execution records supplied to the high-level modules, keeping the executor, verifier library, VLM backend, and evaluation settings fixed.

TaskG2SSimple history
VideoUnmask84%68%
VideoRepick60%68%
VideoPlaceOrder90%82%
ButtonUnmask70%54%
BinFill58%40%
SwingXtimes56%54%
VideoUnmaskSwap24%18%
Average63.10%54.90%

Across these seven RoboMME tasks, full Goal2Skill achieves 63.1% average success, compared with 54.9% for recent execution history, an improvement of 8.2 percentage points. This comparison evaluates our complete memory configuration, including its retention strategy, rather than memory organization alone.

These experiments assess the contributions of verification and memory within our framework. They are not reproductions of RoboCerebra or PrediMem, and we do not claim direct performance superiority over either framework. We have revised our paper to incoporate the additional results.

AuthorsOpenReview ↗

Response to Reviewer YRtf

W3: Real-robot evaluation.

Thanks for your great suggestion. We agree that real-robot validation is important. Our current evaluation uses RoboMME and RMBench to study memory, progress verification, and recovery under systematic and reproducible conditions. These benchmarks support our conclusions about performance on the evaluated simulated tasks, but do not establish effectiveness in physical deployment.

Real-robot experiments are not included due to the limited resources of physical deployment. We will state this limitation explicitly and identify physical-robot validation as future work.

W4: Progress assessment under ambiguous observations.

This is a good point. We agree that task-specific checks can make incorrect judgments when observations are ambiguous. Goal2Skill addresses this through conservative progression decisions and experience-driven refinement of its verification procedures.

As described in Section 3.2.3, when the verifier cannot establish either completion or a failure requiring intervention, it returns ONGOING, retaining the active subtask. It combines current observations with task memory and, where necessary, evidence across multiple checkpoints; ambiguous or history-dependent cases invoke VLM reasoning. These mechanisms reduce reliance on a single observation.

Moreover, verifier self-evolvement addresses failure patterns identified from execution experience, with candidate updates validated through trajectory replay before adoption. For example, the swing-counting case in Section 5.3 initially infers progress from planned subgoal transitions, which can diverge from actual physical execution. The refined procedure instead detects arrivals at the left and right target regions, suppresses duplicate detections, and counts a cycle only after observing the required transition sequence. This grounds progression in execution evidence. To quantify the contribution of this refinement, we compare an initial generic VLM-based verifier, initialized without manually authored task-specific rules, with the self-evolved verifier on 50 episodes per task across six RoboMME tasks.

TaskInitial verifierEvolved verifierChange
PickXtimes74%92%+18 pp
ButtonUnmaskSwap8%50%+42 pp
VideoRepick70%60%−10 pp
VideoPlaceOrder82%90%+8 pp
PickHighlight64%76%+12 pp
VideoPlaceButton22%52%+30 pp
Average53.30%70.00%+16.7 pp

The refined verifier improves success on five of the six tasks, increasing average success by 16.7 percentage points. Complementing the verification comparison in W2, this result supports the practical value of experience-driven refinement for end-to-end task execution.

Thanks for your additional experiments, and I would maintain my original rating.

Reviewer_h7BROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Goal2Skill proposes a hierarchical framework for long-horizon robotic manipulation. A high-level VLM handles task decomposition, memory management, progress verification, and reflective correction after failures, while a low-level policy executes actions with task-conditioned visual filtering. The system uses evidence from actual execution to decide whether to continue the current subtask, advance to the next one, or replan. It also introduces a self-evolving verifier that updates verification skills using historical trajectories and feedback. The paper evaluates overall performance on RoboMME and RMBench and provides selected ablations of the memory and reflection modules.

Critical strengths
  1. The system is fairly complete and addresses practical issues during execution. In addition to planning, it retains historical information, checks whether tasks have been completed, and handles recovery after failures, forming a comprehensive execution process.
  2. The experiments are relatively extensive and support the effectiveness of the overall system. Evaluation on two benchmarks includes comparisons against multiple baselines, selected module ablations, and case studies. The results on the full RoboMME benchmark also show an overall improvement, and the breadth of evaluation is a strength.
Critical weaknesses
  1. The methodological novelty is unclear. The paper presents a comprehensive pipeline for planning, memory, verification, and error recovery. However, the descriptions of proactive memory and multi-level recovery remain high-level, leaving their key advances over existing dual-system methods unclear.
  2. The self-evolving verifier is insufficiently specified and evaluated. The processes for generating, updating, and selecting verification rules are unclear. The paper also lacks a controlled comparison with a fixed verifier, making it difficult to assess its degree of automation and practical benefits.
Questions for authors
  1. What specific mechanisms distinguish proactive memory and multi-level recovery from those used in existing dual-system methods?
  2. How are verification rules generated, updated, and validated before being added to the library? Does this process involve any manual editing or selection?
  3. How does performance change when the verifier is kept fixed, with all other components and evaluation settings unchanged?
2 comments
AuthorsOpenReview ↗

Response to Reviewer h7BR

Thank you for your valuable comments and for recognizing the system’s completeness and experimental breadth. Below we address the raised points one by one.

W1 / Q1: What distinguishes proactive memory and multi-level recovery from existing methods?

Goal2Skill introduces a unified high-level coordination framework in which planning, memory, verification, and recovery jointly govern task progression. We do not claim that hierarchical planning or memory is individually new; our contribution lies in the following mechanisms and their interaction.

  • Plan-aware memory for verification and recovery. Memory-augmented policies such as MemoryVLA, MEM, and ReMem-VLA retain historical context to support action generation. Goal2Skill explicitly conditions memory updates on the unfinished plan, preserving information needed by future steps. Episodic history, working memory, and the error register serve complementary needs: cross-subtask retention, local progress tracking, and recovery from previous failures. This organization allows a shared framework to accommodate different task demands.
  • Evidence-guided recovery at different scopes. Related approaches such as CheckVLA, Phoenix, and PhysReflect-VLA study execution verification and corrective feedback. Goal2Skill connects this feedback to task-level decisions: RETRY repeats the current operation, ADJUST modifies its local specification, and REPLAN revises the unfinished sequence while preserving verified progress that remains valid. Verification determines whether the intended outcome occurred; reflection determines what should change. Our verifier also refines its checking rules with self-evolvement, improving the criteria that govern subsequent task progression and recovery.
  • Coordination across three timescales. Frequent action execution, periodic verification, and sparse planning are coupled through shared memory and verification outcomes. Confirmed completion advances the existing plan without regenerating it; failures trigger targeted correction. The self-evolving verifier further refines the procedures governing these transitions through execution experience.

Our goal is a general coordination framework that accommodates diverse task requirements, while allowing the usefulness of individual components to vary across tasks. Below we futher compared our model with a simple history-based memory baseline, which replaces the original memory with the last eight execution records to help the high-level modules make decisions. Goal2Skill achieves 63.1% average success across seven RoboMME tasks, compared with 54.9% for simple history. These results support the proposed configuration across the evaluated tasks. We will clarify this methodological positioning in the revision.

TaskG2SSimple history
VideoUnmask84%68%
VideoRepick60%68%
VideoPlaceOrder90%82%
ButtonUnmask70%54%
BinFill58%40%
SwingXtimes56%54%
VideoUnmaskSwap24%18%
Average63.10%54.90%
AuthorsOpenReview ↗

Response to Reviewer h7BR

W2 / Q2: Regarding verification rules generated, updated, and validated?

Thank you for this question. "Self-evolving" refers to the model revising verification rules from execution experience, rather than necessarily generating a verifier entirely from scratch. The framework supports either human-provided seed rules or automatic initialization. In our experiments, we use automatic initialization without manually authored task-specific rules: the VLM directly assesses completion from the subtask-start and current observations, subtask requirements, and relevant memory, returning a completion judgment, confidence, and supporting evidence. This generic VLM-based check serves as the initial verifier, whose checking procedures are subsequently refined through execution experience.

In particular, for verifier development, we split the original training set into new training and development subsets at a 9:1 ratio, keeping the formal test set separate. The high-level model performs five rounds of cross-episode refinement. It examines execution trajectories from the new training subset to identify incorrect progress judgments or premature advancement, then proposes revisions to existing verification rules or additional verification skills. Candidate skills are checked through trajectory replay: their progress judgments and advancement timing are compared against the ground-truth trajectories, and candidates passing consistency checks are added to the library.

After the five development rounds, the library is fully frozen throughout formal evaluation. Formal test trajectories are not used to revise the library, and no updates carry over between test episodes. Ground-truth trajectory feedback is used to validate candidate updates during development; this is distinct from the observation- and memory-based verification performed during test execution. We have revised the paper to better clarify this.

Q3: Do the evolved verification rules improve performance when other components are held fixed?

We further included a controlled comparison starting from the same seed verifier to show the effectiveness of our module. One configuration retains the initial rules, while the other uses the rules obtained after the five refinement rounds described above. Both verifiers are frozen during evaluation. Other components are held fixed, so the only difference is the verification rules. Each configuration is evaluated on 50 formal test episodes per task, separate from the scenarios used for rule refinement.

TaskInitial verifierEvolved verifierImprovement
PickXtimes74%92%+18 pp
ButtonUnmaskSwap8%50%+42 pp
VideoRepick70%60%−10 pp
VideoPlaceOrder82%90%+8 pp
PickHighlight64%76%+12 pp
VideoPlaceButton22%52%+30 pp
Average53.3%70.0%+16.7 pp

The evolved verifier improves success on five of the six tasks, increasing average success by 16.7 percentage points. These results provide evidence that the model-generated rule refinements benefit subsequent independent closed-loop executions, beyond the trajectories used to develop them. We have revised the paper and incorporated the results.

Reviewer_Ve4bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies long-horizon manipulation with a VLM planner and a VLA executor. The planner breaks the task into subtasks, keeps a record of what has happened, and checks the outcome of execution before moving on. If something goes wrong, it can retry, adjust the current subtask, or revise the unfinished part of the plan. The memory contains an episodic history, working memory, and an error register. The remaining plan is used to decide what information should be retained for later steps.

The paper also introduces a self-evolving verifier. Verification procedures are kept in a skill library and are updated using execution feedback, with proposed changes checked through trajectory replay. The experiments cover RoboMME and five RMBench tasks. The reported improvements are largest on tasks requiring the robot to retain earlier observations or interaction outcomes.

Critical strengths

S1. The problem is well motivated. In a long task, issuing the right action is not enough; the robot also needs to know whether it worked before attempting the next step. The example of scanning the can explains this nicely. I like that a correction can leave earlier successful steps intact.

S2. The paper is readable, and the examples are useful. In particular, the distinction between checking what happened and deciding how to recover is easy to follow. The appendix gives fairly detailed training configurations for the low-level policy.

S3. The results on the complete RoboMME benchmark are encouraging: 51.5% average success compared with 43.6% for the strongest non-oracle baseline. I appreciate that the authors include the full results, since the six tasks in the main table give a rather different impression of the size of the improvement. The RMBench results also show benefits on tasks where previous attempts matter. Appendix C.2–C.3 also discusses where FrameSamp remains stronger, which helps put the overall improvement in perspective.

Critical weaknesses

W1. My main reservation is that I do not understand how the verifier evolves in practice. Section 3.2.3 says that the system identifies gaps from trajectories, updates skills, and validates the updates through replay. I can follow the intention, but not the procedure. For example, in Figure 4 the system replaces counting subgoal transitions with a check based on actual arrivals at the left and right regions. This is a good change. What I cannot tell is who or what produced it. Was the counting procedure generated by the VLM? Were the regions and the rules supplied by the authors? How was the change judged to be better?

This matters for the novelty of the work. The paper discusses several existing approaches that already combine high-level planning with low-level execution, memory, or recovery. I do not think combining these ideas is uninteresting, but the self-evolving verifier seems to be where the paper wants to go beyond that combination. At present, it is also the part I find hardest to assess.

W2. I think the experiments show that the system can help, but they leave several questions about what accounts for the improvement. I have separated these below so that the authors can respond to them individually.

W2.1. There is no fixed-verifier comparison. The example in Figure 4 shows the advantage of a better check, but does not show the advantage of the process that produced it. A fixed verifier supplied with the corrected counting rule would presumably avoid the same mistake. It would be useful to start with the same library, allow one version to update it, and compare the two on separate executions. Otherwise I cannot tell how much the evolution mechanism contributes to the reported results.

There is a related missing detail: which trajectories are used to make these updates? I could not find whether the library is built before evaluation, changes within an episode, or carries changes from one evaluation episode to the next. What feedback is available for replay validation is also unclear. These are different experiments, and I would need to know which one was performed to interpret the numbers.

W2.2. The comparison with GroundSG+QwenVL would be more informative if the low-level setup and VLM backend were matched. Goal2Skill uses visual filtering and masked versions of 20% of the demonstrations, as described in Section 3.3 and Appendix A.1–A.2. It is good that the total number of demonstrations is unchanged. Still, the inputs and training treatment have changed, so some of the gain could come from these choices. I would like to see how much remains when the hierarchical baseline has the same visual treatment and low-level policy. This seems more useful for understanding the proposed planner and verifier than adding more comparisons with standard action policies.

W2.3. I am less convinced that the memory ablation establishes the benefit of the three-part organization. Table 5 removes the components cumulatively. After removing the error register, removing working memory changes the average only from 28.0% to 27.7%; Blocks Ranking Try improves from 42% to 54%, whereas Rearrange Blocks drops from 36% to 21%. Appendix C.1 acknowledges this and suggests that working memory helps track evolving intermediate states. That is a reasonable interpretation, but this experiment cannot separate the effect of working memory from its interaction with the error register, which is already absent. The large drop after also removing episodic history supports keeping a history, but does not by itself show that splitting it into these three components is preferable. A simpler history-based memory would be a useful reference here.

Overall, I think there is a useful system here. My difficulty is with the gap between the claim about improving verification and the evidence provided for it. The benchmark gains alone do not answer that question, especially when the low-level setup also changes. I would need a clearer account of the update procedure and the comparisons in W2.1–W2.2 to recommend acceptance.

Questions for authors

Questions:

Q1. Could the authors explain the actual sequence of steps that produced the change in Figure 4, including any human input, the trajectories used, and the feedback used to accept the update? Please also clarify whether the library was frozen during the reported evaluation or changed across evaluation episodes.

Q2. Can the authors report the fixed-library comparison described in W2.1? I am particularly interested in whether the updated checks help on executions other than those used to develop them.

Q3. For the comparison in W2.2, were GroundSG+QwenVL and Goal2Skill given the same low-level checkpoint, visual processing, and VLM backend? If not, could the authors provide a comparison that holds these choices fixed, so that the difference more directly reflects the proposed coordination method?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Ve4b

Thank you for your detailed review and constructive comments. We appreciate your recognition of the motivation, the distinction between verification and recovery, and the encouraging full-benchmark results. Below, we clarify and address the raised questions and concerns.

W1 / Q1: How does the verifier evolve?

This is a good question. "Self-evolving" refers to the model revising verification rules from execution experience, rather than generating a verifier entirely from scratch. Concretely, the system starts with a seed rule, which can be a generic default check or incorporate human-provided prior knowledge. In our experiments, the manually supplied seed is implemented by directly asking the VLM to assess completion from the subtask-start and current observations, the subtask requirements, and relevant memory, returning a completion judgment, confidence, and supporting evidence.

For verifier development, we split the original training set into new training and development subsets at a 9:1 ratio, keeping the formal test set separate. The high-level model performs five rounds of cross-episode refinement. It examines execution trajectories from the new training subset to identify incorrect progress judgments or premature advancement, then proposes revisions to existing verification rules or additional verification skills. Candidate skills are checked through trajectory replay: their progress judgments and advancement timing are compared against the ground-truth trajectories, and candidates passing consistency checks are added to the library.

In Figure 4, the relevant change is from inferring swing counts through nominal subgoal transitions to checking actual arrival events. The revised procedure requires the appropriate sequence of right- and left-region arrivals before incrementing the count and suppresses repeated detections of the same arrival. This grounds advancement in observed execution rather than assuming that a requested movement was completed.

After the five refinement rounds, the verifier module is frozen during test time. Formal test trajectories are not used to revise the rules, and no skill updates occur within or across test episodes. We have revised Section 3.2.3 to clarify this.

W2.1 / Q2: Do the evolved verification rules improve independent executions?

Following your suggestion, we added a controlled comparison starting from the same seed verifier. One configuration retains the initial rules, while the other uses the rules obtained after the five refinement rounds described above. Both verifiers are frozen during evaluation. Other components are held fixed, so the only difference is the verification rules. Each configuration is evaluated on 50 formal test episodes per task, separate from the scenarios used for rule refinement.

TaskInitial verifierEvolved verifierImprovement
PickXtimes74%92%+18 pp
ButtonUnmaskSwap8%50%+42 pp
VideoRepick70%60%−10 pp
VideoPlaceOrder82%90%+8 pp
PickHighlight64%76%+12 pp
VideoPlaceButton22%52%+30 pp
Average53.3%70.0%+16.7 pp

The evolved verifier improves success on five of the six tasks, increasing average success by 16.7 percentage points. These results provide evidence that the model-generated rule refinements benefit subsequent independent closed-loop executions, beyond the trajectories used to develop them. We have revised the paper and incorporated the results.

AuthorsOpenReview ↗

Response to Reviewer Ve4b

W2.2 / Q3: Regarding the comparison with GroundSG

Goal2Skill and GroundSG use the same low-level VLA and VLM backend. Goal2Skill also follows RoboMME’s low-level training recipe and uses the same 100 original training demonstrations per task.

The difference in low-level treatment is task-conditioned visual masking. This transforms the image inputs without changing its architecture or adding model parameters. To align training with execution-time filtering, we replace the images in 20 of the 100 demonstrations with masked versions derived from those same demonstrations; the remaining 80 retain their original observations. Thus, the training uses neither additional demonstrations nor an external source of data. During execution, masking is applied only to the low-level policy's inputs, while the high-level modules retain access to the original observations. We will clarify the configuration differences and revise the attribution claims accordingly.

W2.3: Does the three-part memory organization help beyond retaining history?

Thanks for raising this question. To address these concerns, we include independent ablations, removing each component individually while retaining the other two, together with a simple history-based memory baseline. This baseline replaces the original memory with the last eight execution records to help the high-level modules make decisions. The results on seven RoboMME tasks are shown below.

TaskFull G2Sw/o episodic historyw/o working memoryw/o error registerSimple history
VideoUnmask84%84%72%80%68%
VideoRepick60%62%64%60%68%
VideoPlaceOrder90%86%82%86%82%
ButtonUnmask70%54%54%66%54%
BinFill58%56%52%44%40%
SwingXtimes56%42%60%58%54%
VideoUnmaskSwap24%16%24%18%18%
Average63.10%57.10%58.30%58.90%54.90%

Independently removing episodic history, working memory, or the error register reduces average success. Unlike the cumulative ablation, these comparisons measure each component's contribution in the presence of the other two. Futhermore, full G2S achieves 63.1% compared with 54.9% for simple history. This supports the proposed memory configuration over the tested history-based alternative.

These benefits remain task dependent: simple history performs better on VideoRepick, and some component removals improve individual task results. However, Our goal is to provide a general framework that accommodates diverse task demands, while recognizing that different tasks may benefit from different subsets of its memory components. We have revised the paper and incorporated the included results.

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts, particularly the fixed-versus-evolved verifier comparison and independent memory ablations, which substantially address my concerns regarding W2.1 and W2.3.

However, two concerns remain:

  1. Self-evolving verifier (W1): The clarification of the refinement process is helpful, but some implementation details, which I believe is essential for clarity, remain unclear. For example, how are VLM-generated verification rules converted into executable procedures, and what specific criteria are used to accept updates during trajectory replay? A brief clarification would help establish the reproducibility of the central contribution.

  2. Baseline fairness (W2.2): I appreciate the clarification of the baseline configurations. While I consider the overall system-level comparison reasonable, the differences in visual masking and low-level training still make it difficult to isolate the contribution of the proposed high-level coordination mechanism. I would still encourage the authors to provide a controlled comparison using the same low-level checkpoint and visual processing for GroundSG+QwenVL and Goal2Skill, if feasible. Otherwise, I hope this limitation can be explicitly acknowledged in the revised paper.

Overall, the rebuttal has addressed several of my concerns and strengthened the paper. But I would maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.