acceptodds
Under review as a conference paper at ICLR 2027

SYMBOLIC COMPOSITIONAL PLANNING FROM DEMONSTRATED EXPERIENCE FOR EMBODIED AGENTS

Abstract

Vision-language models (VLMs) exhibit strong in-context learning capabilities, yet embodied agents built on them struggle with complex tasks given only a few demonstrations, often omitting critical steps, violating action preconditions, or organizing the overall workflow incorrectly. These failures stem from an incomplete understanding of demonstrated action dependencies and a failure to distinguish reusable local procedures from task-specific workflow structure. To address this problem, we propose an embodied planning method that combines trajectory composition with symbolic analysis, reusing demonstrated local procedures while reorganizing them into workflows for new tasks. Demonstration fragments preserve local action structure, while symbolic analysis makes their preconditions and effects explicit to guide composition and connections between fragments. During execution, the agent maintains scene memory and uses execution feedback to revise the remaining plan. Experiments on EB-ALFRED and EB-Habitat show success-rate improvements of 21% and 22 %, respectively, over standard few-shot planning with the same backbone.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied agents, vision-language models, task planning, neuro-symbolic reasoning, trajectory composition, in-context learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_e4hSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents COPE, a framework for in-context adaption of VLAs. It aims to address prior work limitations that do not work well on complex, multi-step tasks due to improper action precondition resolution or incorrect subtask ordering. The proposed method first organizes relevant demonstration fragments for the subtasks on the global level, then adapt the organized sequence to the execution. In the first part, it uses a symbolic analyzer to validate the preconditions and the results of the demonstration fragment. In the latter part, a closed-loop grounding mechanism guides the trajectory composer to generate valid actions that can be connected together. The authors performed experiments on EB-ALFRED and EB-Habitat against 4 baseline methods. The idea is innovative, but the evaluations need further work.

Critical strengths
  • The idea in Section 4.1 to treat long horizon demonstrations as procedural fragments followed by analysis and ordering is innovative.

  • Enforcing subtask validity using symbolic analysis avoids the typical hallucinations in pure VLMs.

  • The ablation results in Table.4 supports the effectiveness of the proposed fragment reuse and symbolic analysis mechanisms.

  • The paper is well very clear.

Critical weaknesses
  1. In Section 5.1, it is not necessary to compare against standard VLM planning. Having them in the table is ok for reference purposes, but they do not suggest that the proposed method is advantageous because those standard VLMs have less human-tuned reasoning prior in them. Secondly, when comparing against the other training-based/free methods, the authors should fix the same VLM backbone. Otherwise, it would be hard to tell if the difference is due to the VLM or due to the proposed innovations.

  2. The authors should provide a time complexity analysis regarding the steps described in Section 4.1. The fragment selection, for example, does it have to be re-run as the task executes? And for how often?

  3. Eq.2 contains an unexplained complex function ϕj\phi_j. Is there a bijective mapping from the demonstration task to the target task?

  4. In Table.4, it is recommended to also ablate the visual verifier, which can isolate the contribution of the visual grounding.

Questions for authors
  1. Line 148: how is the rule set R defined? Does it come from human annotation or simply just simulator state flags?

  2. Line 207: is there an underlying assumption that a relevant fragment can always be found? What happens if one cannot be found?

  3. Line 210: it is recommended to use another variable to denote demonstration id. iji_j could lead to confusions since i and j are usually used pairwise.

  4. Line 244: how does the VLM guarantee the correctness of the fragment ordering?

  5. Could the authors give examples of the failure cases and explain them? Are they due to mismatched demonstration, or any other issues.

Reviewer_GnvYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes COPE, a training-free VLM-based embodied planning approach that leverages demonstrations to generate task plans. The method consists of two stages: trajectory composition and closed-loop execution. In the first stage, COPE translates the task instruction into goal predicates and uses them to retrieve relevant trajectory segments from the demonstrations. The target object categories in these segments are then adapted to the current task. Subsequently, COPE determines the ordering of the retrieved segments and infers their preconditions and effects based on pre-defined action rules. When inconsistencies arise between consecutive segments, connecting actions are generated and inserted. In the second stage, the agent grounds the resulting plan to specific object instances in the environment and employs a VLM-based search planner and visual verifier to locate target objects and refine subsequent actions as needed. Experiments on EmbodiedBench demonstrate that COPE outperforms previous training-free methods, with ablation studies examining the contributions of its components.

Critical strengths

Extracting relevant segments from demonstration trajectories and explicitly modeling action preconditions are well-motivated design choices in planning.

Critical weaknesses
  1. The proposed approach relies on a series of carefully designed demonstration processing steps, many of which may be brittle, benchmark-specific, or dependent on strong assumptions. For example, the method uses goal predicates as queries to retrieve relevant demonstration segments. This requires the agent to accurately infer the task goals from language instructions alone, without prior knowledge of the environment, which may be unreliable or even infeasible. Moreover, the approach appears to assume that goal predicates can be addressed independently without complex interdependencies, and that relevant demonstration segments are available for each type of goal predicate. Another strong assumption is that a pre-defined set of action rules is available and is able to infer the preconditions and effects of arbitrary action sequences. If both goal predicates and action models are already available, it is worth considering whether conventional symbolic planning algorithms could provide a simpler and better alternative.

  2. The experiments also leave considerable room for improvement. First, the results in Table 1 appear to be a subset of those in Table 2, making it unclear why they are presented separately. Also, the improvements of COPE over prior methods in Table 1 are relatively modest. Second, the few-shot results with Qwen3-VL-32B-Instruct in Table 3 are substantially worse than the corresponding results in Table 2, which is somewhat counterintuitive. Finally, the results in Table 3 suggest that the effectiveness of COPE is highly sensitive to the selection of demonstrations, with a strong reliance on in-domain examples. This raises concerns about the method's robustness and generalizability to tasks or environments for which closely matched demonstrations are unavailable.

Questions for authors

Where are the demonstrations used in the main experiments sourced from, and how many demonstrations are provided? Are the same demonstrations used across different evaluation splits? Are the baseline methods given access to the same set of demonstrations and the same predefined action rules to ensure a fair comparison?

Have the authors considered a simpler baseline that directly incorporates the retrieved trajectories (or segments) and relevant action rules into the VLM's context, without performing the proposed composition, connection, and refinement procedures?

Reviewer_8WMgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes COPE, a framework for few-shot, high-level embodied task planning. A VLM converts a task instruction into goal predicates, retrieves contiguous action fragments from successful demonstrations, and adapts them to the target objects. A rule-based symbolic analyzer derives and checks fragment preconditions and effects, guiding fragment ordering and the insertion of connecting actions. During execution, the agent grounds object references using observations and makes local adjustments after search or verification failures. Experiments on EB-ALFRED and EB-Habitat report improved task success over direct planning with the same VLM backbone.

Critical strengths

Coherent and practically motivated design. The distinction between reusable local procedures and global workflow composition is clear. Explicit symbolic checks of preconditions and effects help prevent invalid action sequences.

Promising results with component-level evidence. COPE improves average success rates by 21 and 22 percentage points over direct Qwen3-VL-32B planning on EB-ALFRED and EB-Habitat, respectively. Table 4 also indicates that fragment reuse and symbolic analysis contribute to the gains

Critical weaknesses

Limited evidence for methodological distinctiveness. The framework brings together familiar ideas from demonstration reuse, modular VLM planning, symbolic verification, and feedback-driven recovery. I recognize that Table 4 supports the value of fragment reuse and symbolic analysis within COPE. However, these ablations do not establish the advantage of the overall design over a stronger modular agent planner equipped with comparable action rules, feedback, replanning, and inference resources. This makes it difficult to assess how substantial the proposed methodological contribution is.

An apparent baseline attribution problem in Table 2. The five EB-ALFRED results attributed to ELITE in COPE Table 2 exactly match the base Qwen2.5-VL-72B-Instruct row in the original ELITE paper's supervised-setting Table 2, rather than its full ELITE method. The full ELITE method reports higher results in that setting. COPE Table 1 appears to draw on ELITE's separate online-setting results, so the issue is specific to Table 2 and the comparison protocol. This is important because it affects the interpretation of the reported advantage over a closely related method.

Insufficient clarity on evaluation settings and total inference cost. The Qwen3-VL-32B Long Horizon baseline is 48% in COPE Tables 1-2 but 38% in Table 3. These figures may reflect different demonstration sets, yet the relationship between their evaluation settings needs to be explained. Moreover, equal per-call output limits and environment-action budgets do not guarantee comparable total inference cost for a system making multiple planning, verification, and recovery calls.

Questions for authors

Could the authors clarify the source of the ELITE numbers in Table 2? The listed values match the base Qwen2.5-VL-72B results, whereas the complete ELITE method achieves higher success rates in the original paper's supervised setting. Which ELITE configuration and evaluation protocol were used for this comparison?

What accounts for the Qwen3-VL-32B baseline's 48% Long Horizon success rate in Tables 1-2 versus 38% in Table 3? Please specify the demonstration set and other relevant evaluation conditions for each result.

Beyond the existing component ablations, what evidence distinguishes COPE from a comparably equipped modular agent planner using structured prompts, symbolic checks, and replanning? It would also help to report the average total VLM calls/tokens or runtime, since these costs may differ considerably from direct VLM planning even when the interaction budgets are the same.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.