acceptodds
Under review as a conference paper at ICLR 2027

SMOOTH OPERATOR: IN-PLACE SYMBOLIC PLAN REPAIR VIA CONTINUAL OPEN-WORLD PREDICATE AND OPERATOR INVENTION

Abstract

Open-world embodied agents frequently fail during execution because predefined symbolic planning domains are fundamentally incomplete. Existing recovery methods address unmet preconditions or planning impasses by re-planning globally from scratch, causing severe plan churn. In contrast, we tackle failures where nominal preconditions are met, yet physical execution stalls and the intended action effects are not observed. We propose SMOOTH (Symbolic Model Open-World Online Repair through Tested Hypotheses), a neuro-symbolic framework for online, in-place plan repair and continual domain induction. Upon failure, SMOOTH leverages a vision-language model to jointly induce open-vocabulary relational predicates and synthesize novel operators composed from primitive skills. Candidate abstractions are physically verified via counterfactual simulation rehearsals to avoid un-grounded proposals. Verified compensatory actions are spliced directly at the failure point, preserving the valid downstream plan suffix and minimizing plan disruption. Discovered abstractions are consolidated into persistent memory for lifelong reuse. Evaluations on OpenRepairBench and Mini-BEHAVIOR show that SMOOTH achieves higher task success and domain reuse than state-of-the-art baselines, while our ablations highlight the critical roles of in-place path preservation and physical validation.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neuro-Symbolic AI, Robot Manipulation, Task and Motion Planning, Plan Repair, Continual Learning, Vision-Language Models, Predicate and Operator Induction, PDDL

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_ves2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SMOOTH is a framework for repairing a robot’s symbolic plan in place when a skill fails even though its modeled preconditions are satisfied. Instead of globally replanning, it diagnoses the missing condition, invents a new predicate, synthesizes an operator to establish it, verifies the repair physically on a checkpointable simulator/digital twin, splices the repair before the failed step, and persists the new knowledge for future tasks.

Core contributions Online in-place plan repair. SMOOTH splices a compensatory repair segment directly before the failed action and keeps the remaining plan suffix intact. It verifies symbolic consistency and physically rehearses the repair, avoiding global replanning churn.

Failure-driven predicate discovery. When an action fails despite satisfied preconditions, a VLM proposes rival open-vocabulary predicates. SMOOTH tests them via directed physical interventions: a predicate is accepted only if satisfying it makes the failed skill succeed.

Operator synthesis via skill chaining. To establish the new predicate, SMOOTH reuses an existing operator, binds a named skill, or composes primitive skills into a sequence. It derives the PDDL operator schema and validates it physically, avoiding sample-inefficient RL.

Persistent abstraction memory. Discovered predicates, grounding classifiers, and composed operators are stored long-term and reused zero-shot across later tasks.

Validation The paper evaluates SMOOTH on:OPENREPAIRBENCH: 77 long-horizon manipulation tasks across Kitchen, Study Desk, and Clinic domains in a cuRobo-backed digital twin.

Critical strengths
  1. The system invents both predicates and executable operators, and fixes the plan in place instead of globally replanning. The key mechanism is “VLM proposes + physical hypothesis testing”—the VLM only suggests candidate predicates, and a candidate is accepted only if a physical intervention makes the failed skill succeed. Ablation shows this isn’t just for show: removing physical validation drops turn-1 success from 100% to 26.9%.

  2. New predicates become executable classifiers, new operators become PDDL schemas, and skill bindings plus composition recipes are written into JSON memory. Later tasks can plan and reuse them directly, and after convergence, it can run with zero VLM calls.

Critical weaknesses
  1. Lack of closed-loop validation of the core repair pipeline. First, there is no real-robot closed-loop validation of the physical hypothesis testing pipeline, and the symbolic-to-execution loop is also incomplete. The physical test depends on a rollback-able digital twin; real-world rollback is often impossible, and rehearsal is skipped when the gripper holds an object—precisely when grasping, moving, and placing are most failure-prone. The paper reports only a preliminary real-hardware transfer test on a Pantry pick-and-place task, not a closed-loop repair, and no detailed quantitative real-robot results; Appendix D is offline analysis, so sim-to-real validity remains unverified. Moreover, operator synthesis only binds to existing named skills or compositions of primitives; target poses and affordances are assumed to be provided by the skill library, not learned or validated. If a newly invented predicate requires a novel affordance or motion strategy, SMOOTH cannot guarantee it will be converted into a real executable action, leaving the execution layer open.

  2. Optimistic bias and comparison fairness concerns in the evaluation. Headline numbers use best-of selection over 3 seeds and 56–57 configurations (best 95.8% vs. pooled 54.5%). The physical ablation uses only 26 tasks, 1 seed, and some subgroups have as few as 2–4 tasks. In Table 4, SMOOTH uses a best GPT-4 configuration while global replanning uses Qwen3-VL-8B; LASP/CLIMB are re-implemented, and task sets differ for Lu et al. These issues make the claim of significant superiority not fully supported.

Questions for authors

When a newly invented predicate cannot be established by any existing operator, named skill, or composition of primitives, what does SMOOTH do? Does it learn a new skill, or does the repair fail? How often does this happen in the experiments? Does operator synthesis include geometric parameters such as grasp points, placement poses, and approach directions, or does it assume the skill library already provides them?

In many real-world settings, arbitrary rollback is impractical, and rehearsal is skipped when the gripper holds an object. How does physical hypothesis testing work on a real robot in practice? If it is not performed on the real robot, how do you ensure the repair is valid?

In Table 5, the best configuration reaches 95.8%, but pooled success is only 54.5%—a gap of over 40 points. Can you report per-scenario average success under a single fixed configuration (e.g., the default), without best-of selection? Does best-configuration selection significantly exaggerate the method’s advantage?

In Table 4, SMOOTH uses a best GPT-4 configuration while global replanning uses Qwen3-VL-8B, so the model capabilities are not comparable. Can you re-run all baselines with the same VLM backbone and the same task set? How were the LASP/CLIMB re-implementations validated against the original papers’ performance?

How do you manage growth, conflicts, redundancy, and forgetting in the knowledge library over long-term runs?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ves2: Comparison Fairness and Closed-Loop Validation (1/3)

Thank you for the most detailed review we received. Your second weakness was correct on every count and led us to re-run the entire evaluation. Your first weakness is partly fixed and partly a limitation we now state in the main text. References are to the revised PDF.

W2. Optimistic bias and comparison fairness.

Issue you raisedSubmissionRevision
Best-of selectionBest of 56–57 configurations per scenario; 95.8% best vs 54.5% pooledNo selection. One configuration for all tasks (App. A.8, Table 8)
Backbone mismatch in Table 4SMOOTH on GPT-4-class, baseline on Qwen3-VL-8BQwen3-VL-8B-Thinking for every method in Tables 2–6 (§5.1)
Task-set mismatch15 vs 52 tasks (Table 3); 8 scenarios vs 26 tasks (Table 4)Same 62 tasks for every method
Physics ablation size26 tasks, 1 seed, subgroups of 2–462 tasks × 5 seeds = 310 runs per arm; groups of 100–110 runs (Table 6)
Re-implemented baselinesNot validatedReproduced on original benchmarks with original models (App. E)

One change to flag: the compound baseline in Table 4 is now VLM + Lu et al. (a VLM proposes the missing precondition, Lu et al. synthesizes the operator; App. E). Replanning with SMOOTH's own learned model is compared separately in Table 5.

W1. No closed-loop validation of the core repair pipeline. Three sub-points:

Sub-pointOur answer
No real-robot closed-loop validation; sim-to-real unverifiedCorrect, and unchanged. Repairs are computed only in the twin; the repaired plan is executed on the robot skill by skill, with no hypothesis test or rehearsal on hardware (App. H). We made this choice deliberately: as you also point out, physical validation may not be reversible, which limits exploration. We therefore perform hypothesis testing in a twin that supports checkpointing and rollback, and execute only the final validated candidate in the real world.The two pantry tasks are qualitative demonstrations. §6 now says this in the main text
Rehearsal skipped while the gripper holds an objectFixed: checkpoints now save and restore the grasp, so hypothesis tests, operator trials and rehearsal run with an object in hand. Evidence in the PDF: access tasks (robot holds an item while a door or drawer is shut; kitchen, desk, clinic) are part of Group 3, where SMOOTH reaches 100% in Tables 2–4. The learned repair is put_down, open_door, pick, with a tail repair close_door before activate (App. G, Listing 7). A skill blocked only by a full hand is re-screened with the hand emptied (App. A.5)
Operator synthesis only binds existing skills or primitive compositions; poses and affordances come from the skill libraryCorrect. This is a design boundary, stated in §6 ("Bounded motion primitive vocabulary"). See Q1

Q1a. When no operator, named skill or primitive composition can establish a new predicate, what does SMOOTH do?

The repair fails for that predicate. No new skill is learned. The fallback order (App. A.5, Algorithms 2–3):

StepAttemptBudget
1Chain of known operators found by breadth-first search, tried in simulationUp to 4 steps, 5,000 states; no VLM call
2VLM-proposed chain of named skills, one operator per skill3 failed chains
3VLM-proposed composition of motion primitives4 attempts
4Discard this predicate and try the next sufficient rivalUntil rivals run out
5Report failure; everything the attempt added is rolled backThe run counts as a failure

Q1b. How often does this happen?

  • Every benchmark task is solvable with the provided skills and primitives, so the benchmark does not measure the case of an inexpressible skill. We do not claim coverage of it.

  • What we do observe: 14.1% of proposed repair segments are rolled back at the local gate (Qwen, App. C.3, Table 11), and one primitive task (desk dial) fails at the gate on all three backbones we tried (App. C.3).

  • A repair that needs a new continuous motion (e.g. tossing, compliant insertion) fails. §6 names a human-in-the-loop fallback and VLA policies as future work.

Q1c. Does operator synthesis include grasp points, placement poses, approach directions?

Grasp points, placement poses, and approach directions are not passed to the operators directly; instead, they are bound to the objects the operators take as arguments. Operator synthesis selects skills and binds them to objects. Geometry is resolved inside the skill layer (App. H.3):

PosesGraspsTrajectory
SimulationGround-truth 6D poses from the simulator stateFrom the skillcuRobo
Real robotGrounding DINO + SAM masks, then FoundationPoseGraspGencuRobo

Primitives are parameterized by a target, e.g. reach(target="door[?appliance]") (App. G, Listing 5).

AuthorsOpenReview ↗

Response to Reviewer ves2: Fallbacks, Real-Robot Testing and Fixed-Config Results (2/3)

Q2. How does physical hypothesis testing work on a real robot? If it is not performed there, how is the repair known to be valid?

  • It is not performed on the real robot. All tests run in the twin (App. H), so an incorrect hypothesis cannot alter the state of the real robot or its environment. The simulator thus acts as a validator, filtering out invalid repairs proposed by the VLM.

  • The validity of the final accepted repair is therefore guaranteed only to the extent that the twin matches the real world. We make no stronger claim; as noted in §6, the pipeline currently assumes perfect state transfer.

  • Partial evidence off the perfect twin, offline: over 340 evaluations of 45 learned predicates, crisp code keeps .932 accuracy under pose and reading noise, and falls to .621 when joint and latent readings are withheld. Routing unobservable readings to a calibrated judge restores .944 (App. D, Table 14).

  • Why test in the twin at all: with no physical check, 71.0% of accepted repairs are invalid (Table 6). Those would otherwise be discovered on hardware.

Q3. Per-group success under a single fixed configuration. Did best-of selection exaggerate the advantage?

Yes, it inflated SMOOTH's absolute numbers. Fixed-configuration results, 62 tasks × 5 seeds, Qwen3-VL-8B-Thinking, success by failure group:

SettingMethodGroup 1 Direct (22 tasks)Group 2 Masked (20)Group 3 Multi-Gate (20)All
Predicate invention (Table 2)SMOOTH100%80%100%93.5%
LASP32%63%18%37.5%
CLIMB32%20%20%24.3%
Operator synthesis (Table 3)SMOOTH97%80%100%92.5%
Lu et al.71%55%55%61.0%
Compound (Table 4)SMOOTH88%80%100%89.3%
VLM + Lu et al.52%52%35%46.5%
  • Size of the inflation: the submission reported 100%, 100% and 95.8% for the three settings. The fixed-configuration numbers are 93.5%, 92.5% and 89.3%.

  • The advantage over baselines holds without selection: +31.5 to +69.2 points of success on identical tasks and backbone.

  • The same configuration on other backbones, turn 1, 186 runs each: Qwen3-VL-8B 88.7%, GPT-4.1-mini 84.9%, GPT-4o 75.3% (Table 11).

  • Where SMOOTH still fails: Group 2, on clinic-relational tasks, where the 8B model misses the probe-in-cradle state (§5.2).

Q4. Re-run baselines with the same backbone and tasks; how were the re-implementations validated?

Re-run: done (table above). Validation: each re-implementation was run on its original tasks with its original models (App. E).

BaselinePublishedOur re-implementation, original modelsSame code, Qwen3-VL-8BTable
LASP (9 cases × 5 runs)42/45 (93%)40/45 (89%; 90% range 80–96%)34/45 (76%)15
CLIMB zero-shot, mean of 3 domains0.370.340.2316
CLIMB few-shot, mean of 3 domains0.800.740.5916
Lu et al., operator discovery (4 domains × 10 runs)37/4035/40 (90% CI 0.76–0.95)27/4017
  • All six CLIMB cells fall inside the published 95% intervals. LASP's published scores fall inside our 90% range. The one exception is Lu et al. on Nut Assembly (9/10 vs 10/10 published).

  • Every baseline loses accuracy when moved to Qwen3-VL-8B. Part of their gap on OpenRepairBench is therefore a backbone effect, which is why SMOOTH is evaluated on that same backbone.

  • Lu et al. with its original RL skill learner solves 9/62 of our tasks (14.5%). We replaced the learner with a pre-formed skill, which is an upper bound for that method (App. E).

AuthorsOpenReview ↗

Response to Reviewer ves2: Baseline Validation and Library Growth (3/3)

Q5. Growth, conflicts, redundancy and forgetting in the knowledge library.

The submission did not study this. The revision adds a library-growth experiment (App. C.4): the kitchen domain is extended to 4–14 appliances, giving libraries of 38 to 102 predicates and up to 64 operators, with plans of about 30 steps. 5 seeds × 5 library sizes × 2 conditions.

Condition (25 runs per row)RetrievalSuccessDuplicates addedReuseVLM callsTwin trials
Known condition (already in the library)Off92.0%100.0%96.0%6.363,383
On100.0%0.0%100.0%2.00372
Novel conditionOff84.0%0.0%88.0%6.444,223
On88.0%0.0%96.0%5.12485
AspectMechanismEvidence or limit
GrowthPredicate retrieval: rank library predicates by relevance to the failed step (reads the step's objects, false at failure, lexical overlap); list the top 24 in the prompt; screen only predicates over the step's objectsWith retrieval, success and calls do not grow from 38 to 102 predicates: 5/5 at every size, 2.0 calls (Table 12)
RedundancyLibrary predicates are screened physically before any new one is proposed (App. A.4); one operator per skill; known operators are searched before the VLM is asked (App. A.5)An honest negative result: without retrieval, all 25 known-condition runs re-invented an existing predicate under a new name. With retrieval: 0 of 25 (Table 13)
ConflictsNothing enters the library without passing the physical tests; a rejected repair removes every predicate, attachment and operator it added (App. A.6); existing operators gain effects only on newly invented predicates (App. A.5); attachments whose operator no longer exists are quarantined at load (App. A.7)No dedicated conflict-resolution experiment
ForgettingNone. The library is append-only; nothing is pruned or evictedLimitation

Two caveats we want to be explicit about:

  • Retrieval is off by default, so the main results (Tables 2–6) were obtained without it (Table 7). Their libraries are kept per task (App. C.4).

  • The growth study is a separate experiment on GPT-4.1-mini with 5 runs per cell. It shows the trend; it is not a long-horizon lifelong deployment.

Reviewer_VM3yOpenReview ↗
Rating
1
Confidence
2(−1)
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

this work proposes SMOOTH, a framework for online symbolic plan repair in open-world environments. It uses vlm to discover missing predicates and synthesize new operators, verifies them and inserts repair actions into the existing plan while preserving remaining steps. It reports improvements in task success, knowledge reuse, and plan preservation.

Critical strengths

The paper addresses a relevant problem in open-world robotic planning by combining predicate discovery, operator synthesis, and local plan repair. The inclusion of physical verification and persistent knowledge reuse is valuable. The reported results suggest improvements over several existing approaches.

Critical weaknesses
  1. Experimental limitations and unclear generalizability. Although the reported results are promising, some comparisons use different task sets or model configurations. This makes the overall performance advantage difficult to assess conclusively. Furthermore, the method relies heavily on a checkpointable simulator with access to internal scene variables, leaving its robustness in less controlled real-world settings insufficiently established.

  2. Insufficient presentation quality and overall polish . The paper appears insufficiently refined for a conference submission. Several figures are cluttered with small text, symbols, and decorative boxes, making them difficult to read. Figure 4 is a representative example: individual symbols and short expressions are placed inside separate boxes Figures 2 and 3 also contain excessive text, limiting their explanatory value.

Questions for authors

Can the authors provide more directly comparable experimental results under matched task sets and model configurations, and further discuss the practical limitations of relying on a checkpointable simulator?

2 comments
AuthorsOpenReview ↗

Response to Reviewer VM3y: Matched Protocol and Fairness Checks (1/2)

Thank you for the direct assessment. We agree with both weaknesses as they applied to the submission. The evaluation was re-run under one protocol, the figures were redrawn, and the simulator dependence is now stated as a limitation in the main text. References are to the revised PDF.

W1a / Q. Comparisons used different task sets and model configurations.

Fixed. Every main comparison now uses the same 62 tasks, the same backbone (Qwen3-VL-8B-Thinking), 5 seeds, 2 turns and one fixed configuration with no selection (§5.1, Table 8).

SubmissionRevision
Tasks per method15 vs 52 (Table 3); 8 scenarios vs 26 tasks (Table 4)62 for every method
BackboneGPT-4-class for SMOOTH, Qwen for a baselineQwen3-VL-8B-Thinking for every method
ConfigurationBest of 56–57 per scenarioOne, fixed
Seeds3 (1 for the physics ablation)5
SettingSMOOTHBaselineGap in success
Predicate invention (Table 2)93.5%, 4.6 callsLASP 37.5%, 5.7 calls; CLIMB 24.3%, 7.3 calls+56.0, +69.2 points
Operator synthesis (Table 3)92.5%, 5.0 callsLu et al. 61.0%, 10.5 calls+31.5 points
Compound (Table 4)89.3%, 7.1 callsVLM + Lu et al. 46.5%, 15.6 calls+42.8 points
Mini-BEHAVIOR (Table 2)91.7%, 3.9 callsLASP 58.3%, CLIMB 18.8%+33.4, +72.9 points

Two further checks on fairness:

  • Baseline fidelity. Each re-implementation was run on its original benchmark with its original models and matches the published numbers: LASP 40/45 vs 42/45 published; CLIMB within the published 95% intervals in all six cells; Lu et al. operator discovery 35/40 vs 37/40 published (App. E, Tables 15–17).

  • Backbone sensitivity of SMOOTH. Same configuration, turn-1 success over 186 runs each: Qwen3-VL-8B 88.7%, GPT-4.1-mini 84.9%, GPT-4o 75.3% (App. C.3, Table 11).

SMOOTH's own success is lower than the submission reported (100%, 100%, 95.8%). That is the result of removing selection, and we consider the new numbers the correct ones.

AuthorsOpenReview ↗

Response to Reviewer VM3y: Simulator Dependence and Presentation (2/2)

W1b / Q. Heavy reliance on a checkpointable simulator with access to internal scene variables.

We fully agree that our current reliance on a pre-authored digital twin constitutes a limitation, particularly regarding the sim-to-real sensor noise gap. We have updated our "Conclusion and Future Work" (Section 6) to explicitly address this boundary.

As the reviewer rightly notes, real-world environments involve sensor errors, occlusions, and visual noise that a perfectly modeled simulator bypasses. Our physical tea-making episode currently serves as a qualitative demonstration of successful sim-to-real logic transfer, mapping crisp, zero-noise logical predicates generated in simulation to separate, pre-defined perceptual classifiers on the real robot (e.g., using YOLO or Grounding DINO).

However, to quantitatively assess this sim-to-real gap, we conducted an offline evaluation (Appendix D) measuring how our synthesized classifiers perform under imperfect, less controlled observations. We evaluated 340 predicate-scene triples under simulated perceptual degradation, yielding two critical insights:

  1. Robustness of Geometric Code: For geometric predicates, our synthesized crisp code remains highly robust. Even when subjected to simulated real-world pose estimation noise (N(0, 1 cm)), the classifiers maintain 97.4% accuracy.

  2. The Latent Variable Gap and Hybrid Recovery: For non-geometric, latent states (e.g., internal appliance states), accuracy predictably collapses to 44.0% in a "Blind" condition where the simulator's internal variables are entirely withheld to mimic unobservable real-world states. To overcome this, we evaluated a hybrid routing approach that falls back to a calibrated VLM judge (Jev) evaluating the physical action history when internal variables are hidden. This hybrid routing recovers a 94.4% overall accuracy without relying on internal simulator readings.

To move beyond the dependency on a pre-constructed simulator and further address sensor noise in hardware deployment, our roadmap includes two key integration phases:

  1. Addressing Sensor Noise: We will incorporate continuous state filtering and Domain Randomization (DR) into the perception stack to better benchmark physical repair quantitatively under real-world perceptual uncertainty.

  2. Automated Twin Generation: We are integrating automated Real-to-Sim-to-Real (R2S2R) pipelines, such as RialTo and Agentic RSR, to dynamically reconstruct interactive 3D digital twins directly from onboard RGB-D failure observations, eliminating the need for hand-authored environments. By offloading the expensive combinatorial search to the simulator and using hybrid classifiers to bridge the unobservable state gap, SMOOTH provides a safe, highly efficient staging ground for recovery, which we will mature via R2S2R pipelines in future work.

Why we still think the simulator is the right place for these tests: without any physical check, 71.0% of the repairs the system accepts are invalid and success falls from 89.3% to 28.1% (Table 6). Those proposals should not be tried on hardware first.

W2. Presentation quality; Figures 2–4 cluttered.

Agreed. Changes:

  • Figures 2–4 were redrawn. Fig. 4, your example, is now one plan timeline with the spliced segment and the gate outcomes below it, in place of per-symbol boxes. Fig. 2 is a single row of five stages; Fig. 3 is three steps. All three follow the tea example of Fig. 1.

  • The experiments section now has one protocol paragraph (§5.1), and Tables 2–4 share the same rows and columns (success, reuse, D_sym, calls by failure group).

  • Limitations moved from the appendix into §6.

  • The appendix opens with a table of contents; method details, prompts, ablations, baseline fidelity and benchmark construction each have their own appendix (A–H).

If specific passages are still hard to follow, we would be glad to revise them.

Reviewer_G91ZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SMOOTH, a framework for repairing symbolic plans when a plan generated from an incomplete model fails during execution. SMOOTH addresses such failures by: 1) inventing predicates that represent missing preconditions and 2) synthesizing or reusing operators that establish those preconditions. It validates candidate repairs through interactions with the environment, inserts the repairs into the existing plan, and retains the learned model for future tasks. Experiments on the authors’ OpenRepairBench and on Mini-BEHAVIOR show improvements over the evaluated baselines.

Critical strengths
  1. Clear problem formulation and a well-aligned approach. I consider Predicate Invention and Operator Synthesis head-on with the purposed problems in this work.
  2. Structured evaluation of distinct repair capabilities. The use of Predicate Invention, Operator Synthesis and the two together are evaluated separately, strengthening the experiment results. This organization helps clarify the method’s capabilities under different forms of model incompleteness.
  3. Real-world simulation. The use of simulator interventions and execution rehearsals to assess candidate predicates and repair actions provide evidence that proposed model updates can resolve the observed failure and help reject symbolic operators whose claimed effects cannot be physically realized.
Critical weaknesses
  1. Figures 2 and 3 contain dense text and small code annotations, making the predicate-invention and operator-synthesis procedures difficult to follow. Simplifying these figures and enlarging their labels would improve readability; some annotations in Figure 4 would also benefit from larger text.

  2. Insufficient discussion of closely related work. NL-PDDL [1] appears closely relevant to SMOOTH, yet it is neither discussed in related works nor included in the experiments.

  3. Unclear construction of the initial symbolic models. The method assumes a seed symbolic model that admits an initial plan but leads to execution failure. However, the manuscript does not sufficiently explain how these models are constructed or how their omissions are introduced.

References

[1] Natural Language PDDL (NL-PDDL) for Open-World Goal-Oriented Commonsense Regression Planning in Embodied AI. ICLR 2026.

Questions for authors
  1. How are the seed symbolic models constructed for each evaluation setting? Please specify how predicates, precondition attachments, and operators are selected for omission; which choices are manual versus automated; and what construction or filtering rules ensure that an initial symbolic plan exists but encounters an execution failure.
  2. For the operator-synthesis evaluation on OpenRepairBench, please provide a representative initial domain, initial state, planner-generated plan, and execution-failure trace. Which symbolic operator is withheld, which underlying skills remain available, and why does the initial planner still find a plan despite the missing operator?
3 comments
AuthorsOpenReview ↗

Response to Reviewer G91Z: Figures, NL-PDDL and Seed-Model Construction (1/2)

Thank you for the supportive and concrete review. Each weakness and question maps to a specific change. References are to the revised PDF.

W1. Figures 2–4 are dense and hard to follow.

Agreed. All three were redrawn, each around one running example (the tea task of Fig. 1).

FigureSubmissionRevision
Fig. 2, predicate inventionFour text panels with code annotationsOne row of five stages (VLM, code checks, twin test, filtration, attach) and four candidate predicates passing or failing each stage
Fig. 3, operator synthesisThree text boxes plus a schema boxThree steps: find a route, primitive composition, resulting operator
Fig. 4, local gatePer-symbol boxesOne plan timeline with the spliced segment, and the gate outcomes (commit, tail repair, rollback) below it

W2. NL-PDDL is not discussed or compared.

Thank you for the pointer. NL-PDDL (Liu et al., ICLR 2026) is now discussed in §2 and added as a row in Table 1.

Capability (Table 1 columns)NL-PDDLSMOOTH
Online recovery from execution failureNoYes
Open-vocabulary predicatesYesYes
New operator and skill synthesisNoYes
Memory retentionNoYes
Local in-place repairNoYes
  • NL-PDDL plans upfront, using natural-language PDDL and LLM entailment to handle semantic misalignment between goals and the domain. SMOOTH starts where a plan that looked valid fails physically at execution.

  • It is not an experimental baseline because it has no execution-time recovery loop. Every OpenRepairBench task is built so that the seed model yields a plan and that plan fails during execution (App. F), so there is no step at which NL-PDDL would act. We compare against methods that recover at execution time or extend the domain: LASP, CLIMB, Lu et al.

  • We acknowledge this comparison is qualitative only.

W3 / Q1. How are the seed symbolic models constructed? What is omitted, what is manual, and what guarantees a plan that then fails?

This is now specified in App. F, with a full worked example.

Construction rule (OpenRepairBench). Each task has a true domain that governs execution and a seed domain given to the robot. The seed is the true domain with knowledge withheld:

SettingPredicate in vocabularyPrecondition on the failing operatorEstablishing operatorUnderlying skill
Operator synthesis (Table 3)KeptWithheldWithheldKept, listed in deployment notes
Compound (Table 4)WithheldWithheldWithheldKept, listed in deployment notes
Primitive tasksAs per settingWithheldWithheldRemoved; only motion primitives (reach, push, pull, rotate, grasp) remain

Why a plan exists but fails:

  • The precondition is withheld together with the operator that would establish it. The failing step therefore looks applicable, and the planner needs no missing operator to reach the goal.

  • The simulator follows the true domain, so the step fails when executed.

  • Every task is checked mechanically: the seed model must produce a plan, and executing that plan must fail (App. F).

What the robot is told: deployment notes list objects, observable readings and available skills. They never mention a failure mode, a latent precondition or a fix (App. F).

Manual vs. automated: The task families (atom, numeric, negative, relational, conjunction, access, primitive, transfer, distractor, confound, tie, out-of-span) are designed by hand. Each task's world, true model, seed model and omission are generated from the family template, and the plan-then-fail check is automatic.

Mini-BEHAVIOR: Operators and goals are taken from the benchmark's own action definitions and success checks. A precondition that an action can establish (open, toggled, in hand) is removed from an operator, with the predicate kept in the vocabulary. A case is kept only if the complete model solves the task, the reduced model still plans, and that plan fails at execution. This yields the 16 corrupt-precondition cases of §5.1.

AuthorsOpenReview ↗

Response to Reviewer G91Z: Worked Operator-Synthesis Example (2/2)

Q2. A representative operator-synthesis example: initial domain, initial state, plan, failure trace. Which operator is withheld, which skills remain, and why does the planner still find a plan?

Kitchen atom task. App. F prints this task for the compound setting

True domain (governs execution):

(:action activate :parameters (?a - appliance)
:precondition (and (is_appliance ?a) (gripper_empty) (door_closed ?a))
:effect (and (running ?a) (not (idle ?a)) (not (robot_home))))
(:action close_door :parameters (?a - appliance)
:precondition (and (is_appliance ?a) (gripper_empty))
:effect (and (door_closed ?a) (not (door_open ?a))))

Seed domain (given to the robot):

(:action activate :parameters (?a - appliance)
:precondition (and (is_appliance ?a) (gripper_empty))
:effect (and (running ?a) (not (idle ?a)) (not (robot_home))))
; close_door operator: withheld
ItemValue
WithheldThe close_door operator, and the door_closed precondition on activate
Still availableskills pick, place_in, close_door, activate, wait_cycle, retract_home
Initial state(in_chamber bowl microwave) (door_open microwave) (idle microwave) (gripper_empty) (robot_home) (at cup bench)
Goal(and (heated bowl) (robot_home))
Seed planactivate(microwave), retract_home(), wait_cycle(microwave, bowl)
Failure traceStep 1, activate(microwave), returns failure. Its known preconditions is_appliance(microwave) and gripper_empty both measure true (Table 18). Diagnosis: MISSING
Repairdoor_closed invented by the VLM and verified by Eq. 1; an operator is synthesized on the close_door skill; spliced plan is close_door(microwave), activate(microwave), retract_home(), wait_cycle(microwave, bowl). The predicates are lifted to their argument types, e.g. close_door(?appliance)
Next turnThe model plans close_door from the start; zero repairs

Why the planner still finds a plan: activate does not list door_closed in the seed, so the planner never needs close_door. If only the operator were withheld and the precondition kept, the planner would find no plan. That is a planning impasse, the setting of Lu et al., and not the execution failure we study. It is also why operator synthesis is not run on Mini-BEHAVIOR (§5.3).

Reviewer_G91ZOpenReview ↗

Rebuttal Acknowledgement to Authors

Thank you for the response. My main concern lay within the clarity of the experiments, which is resolved with the explanation of generation method of seed models and a representative example. The effort done to revise images is appreciated. In all, I maintain my initial score which is positive.

Reviewer_H898OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SMOOTH uses a VLM to generate missing predicates when an action fails. It tests these systems in both simulation and the real world, then constructs operators by reusing existing skills or primitives. The verified repair is inserted before the failed action while preserving the original plan suffix. Learned predicates, classifiers, and operators are stored for later reuse. The paper evaluates the framework on OpenRepairBench and Mini-BEHAVIOR.

Critical strengths
  1. The paper proposed a clearly defined problem: an action satisfies its known symbolic preconditions but fails physically. The method introduces both predicates and executable operators to repair failures that the existing planning domain cannot represent.

  2. The method connects predicate discovery, operator construction, and plan repair in a unified framework.

  3. The physical verification component shows a clear benefit. In Table 6, removing physical checks reduces success from 100.0% to 26.9% and increases accepted invalid repairs from 0 to 19 across 26 cases.

Critical weaknesses
  1. For the in-place repair setting, in Table 5, SMOOTH and replanning with the same learned knowledge both achieve 95.8% success under the selected configurations. These results do not show a consistent advantage.

  2. The main baseline comparisons use different task sets and model configurations, which may not be fair enough. Table 3 evaluates SMOOTH on 15 tasks and Lu et al. on 52 tasks. Table 4 compares selected GPT-based SMOOTH configurations on eight scenarios with Qwen-based global replanning on 26 tasks. The reported performance differences therefore cannot be directly attributed to the proposed method.

  3. Equation (1) tests whether making a predicate true allows the failed action to succeed. But when several alternative conditions can enable the action, this update may introduce unnecessary repairs.

Questions for authors
  1. How should the advantage of in-place repair be interpreted?

  2. Can the authors compare the methods using the same tasks, VLM backbone, and configuration protocol?

  3. How does the method distinguish a required precondition from a sufficient repair condition? Can the stored precondition be revised when another valid execution route is found?

4 comments
AuthorsOpenReview ↗

Response to Reviewer H898: In-Place Repair and Matched Comparisons (1/2)

Thank you for the precise review. All three weaknesses pointed at real gaps, and each led to a new experiment or a new test in the method. References are to the revised PDF.

W1 / Q1. Same 95.8% success for SMOOTH and replanning: how should the advantage of in-place repair be interpreted?

You are right that the submission's Table 5 showed no advantage. Two things were wrong with it.

  • Success was the wrong metric. The replan arm plans with the same learned model as SMOOTH (Table 7, in_place_splice), so we do not claim that splicing raises goal success. The claim is plan stability (Fox et al., 2006): how much of the announced plan survives the repair.

  • The old tasks could not show stability either. Their plan suffixes had no alternative choices, so replanning returned the same suffix.

The revision adds 24 controlled tasks that vary how much free choice the suffix contains (§5.1, App. F). Table 5:

GroupTasksSMOOTH D_symReplan D_sym
A. Free choices (items onto free shelves)61.05.0
B. Identical machines85.39.2
C. Access fix + free choices64.07.3
D. No free choice (control)42.52.5
All243.46.6
Plan changed under a different tie-break seed (5 seeds)240/2424/24
Latency per repair0.1 ms360 ms

How to read it:

  • Group D is the control. With no alternative in the suffix, both arms tie (2.5 vs 2.5). The gap appears only when the suffix has choices the planner can re-make.

  • In Group A, replan disruption grows with the number of items (1.0, 5.0, 9.0 for 1, 3, 5 items). SMOOTH stays at 1.0, the single inserted fix (§5.5).

  • Replanning is not reproducible: the plan after repair depends on the planner's tie-break seed in 24/24 tasks. The spliced plan never does.

  • Practical meaning: a teammate or a second robot that coordinated around the announced plan (which shelf, which machine) is not invalidated by the repair (§1).

The same effect shows against the baselines, which all replan globally, on the full 62 tasks: D_sym 1.9 vs 4.2 (LASP) and 3.8 (CLIMB) in Table 2, 1.7 vs 4.4 in Table 3, 1.6 vs 5.2 in Table 4.

W2 / Q2. Different task sets and model configurations.

Agreed, and fixed. The mismatches you list are gone:

SubmissionRevision
Table 3 (operator synthesis)SMOOTH on 15 tasks, Lu et al. on 52Both on the same 62 tasks
Table 4 (compound)SMOOTH: best GPT configuration on 8 scenarios; baseline: Qwen on 26 tasksBoth on the same 62 tasks, both Qwen3-VL-8B-Thinking
ConfigurationBest of 56–57 per scenarioOne fixed configuration (Table 8), no selection
Seeds, turns3 seeds, 3 turns5 seeds, 2 turns (§5.1)
Baseline inputsNot specifiedSame deployment notes, initial scene, RGB failure image and action signatures as SMOOTH; no simulator access (§5.1, App. E)

Results on the matched protocol:

SettingSMOOTH successBaseline successSMOOTH callsBaseline calls
Predicate invention (Table 2)93.5%LASP 37.5%, CLIMB 24.3%4.65.7, 7.3
Operator synthesis (Table 3)92.5%Lu et al. 61.0%5.010.5
Compound (Table 4)89.3%VLM + Lu et al. 46.5%7.115.6

SMOOTH's own numbers dropped from the submission's 100%, 100% and 95.8%. That is the cost of removing selection, and the new numbers are the ones we stand behind. We also validated each re-implemented baseline on its original benchmark (App. E, Tables 15–17).

AuthorsOpenReview ↗

Response to Reviewer H898: Necessity Test for Preconditions (2/2)

W3 / Q3. Eq. (1) only tests sufficiency. How is a required precondition distinguished from a sufficient repair condition?

This was a real gap. In the submission a candidate was accepted once making it true let the action succeed. The revision adds a necessity test (§4.2, App. A.4, Algorithm 1 line 20):

  • Start from the failure state and apply generic edits that keep the candidate false (move joints, flip readings, remove or release objects), up to K = 3 edits at a time, over a budget of 50 scene configurations.

  • Re-run the failed skill after each edit.

  • If the skill succeeds while the candidate is still false (in more than 20% of evaluations, to absorb simulation noise), the candidate is rejected as not required.

The full filter a candidate now passes:

CheckWhat it removesWhere
Sufficiency (Eq. 1)Conditions that do not re-enable the step when made true§4.2
Necessity (new)Conditions the step can succeed without§4.2, App. A.4
Minimality (leave-one-out)Extra members of a conjunctive set, e.g. inside dropped from {hatch shut, latch latched, inside}§4.2
EntailmentProxy conditions, e.g. "tray moved" when "cup moved" is the direct one§4.2, App. A.4
RobustnessClassifiers that disagree with the outcome on 8 scene variationsApp. A.4
Operator viability and rollbackRivals no action can establish, or whose repair breaks a later step§4.3, §4.4

Measured effect: with Qwen3-VL-8B-Thinking, the necessity test accounts for 17.8% of all rejected hypotheses, which is 10.3% of all proposed hypotheses (Table 10, Fig. 7). Each of these candidates had already passed the sufficiency test of Eq. 1.

Q3, second part. Can the stored precondition be revised when another valid execution route is found?

No, and we state this as a limitation.

  • The necessity test is a bounded counter-example search (K = 3, 50 configurations), not a proof. An alternative route outside that budget is not found.

  • Operators are STRIPS, so preconditions are conjunctive. "A or B" cannot be represented.

  • Once committed, a precondition is not relaxed if a later episode reveals another route as it might break the formulation for the original task. Rather a new operator is created with that condition.

The consequence of an over-strict stored precondition is an extra repair step that was physically verified to work. It costs plan length; it does not produce an invalid plan. Revising preconditions from later evidence, with the same intervention machinery, is future work.

Reviewer_H898OpenReview ↗

Reviewer reponse

Thanks to the authors for providing the additional experiment results. The authors' response addresses my original concerns regarding the baseline comparison and provides empirical support for the plan-stability benefits of in-place repair over replanning.

However, my concerns regarding precondition necessity are only partially resolved:

  1. Discrepancy in Test Definition: Section 4.2 and Algorithm 1 state that a candidate is rejected if any counterexample is found, whereas Appendix A.4 and your response require success in more than 20% of evaluations. Could you please clarify which rule was actually implemented in the final code?

  2. Reconciling Table 10 and Figure 7: Table 10 reports 17.8% for “Sufficient, but not necessary” and 0.0% for “Skill succeeds without it.” Could you clarify the exact distinction between these two categories? Furthermore, this seems to conflict with the Figure 7 caption, which states that zero hypotheses were rejected because the skill succeeded without them.

  3. Scope of Claims & Plan Pruning: Passing a bounded counterexample search does not guarantee global necessity or “true physical causality,” so I suggest qualifying these claims in the manuscript. Additionally, storing overly restrictive preconditions can have broader consequences than just adding extra repair steps—it may also prune valid, feasible plans in future episodes where alternative execution routes exist.

Based on the above assessment, I would like to preserve my previous score.

Thank you for your careful review and for catching these issues.

1. Discrepancy in Test Definition You are right—the >20%>20\% threshold is what was actually implemented in the final code. We used this threshold so that simulation noise or marginal contact transitions would not cause us to falsely drop valid predicates. We have updated Section 4.2 and Algorithm 1 to match Appendix A.4 and the implementation.

2. Reconciling Table 10 and Figure 7 Thank you for pointing this out. Figure 7 shows the correct results (10.3%10.3\% of all proposed hypotheses, which is 17.8%17.8\% of rejected hypotheses, were "Sufficient, but not necessary"). The 0.0%0.0\% row in Table 10 and the conflicting sentence in the Figure 7 caption were errors left over from an earlier draft, and we have removed them so that the table, figure, and caption are now consistent.

3. Scope of Claims & Plan Pruning We agree with both points:

  • Scope of claims: A bounded counterexample search (K≤3K \le 3, budget ≤50\le 50) only tests empirical necessity locally and does not guarantee global necessity or true physical causality. We have softened these claims in the Introduction, Section 4.2, and the Figure 2 caption.
  • Plan pruning: Because SMOOTH appends conjunctive preconditions monotonically, an overly restrictive precondition can indeed prune valid alternative plans or cause planning impasses in future episodes. We have added this point to Section 6 (Limitation 4) and noted disjunctive schemas and precondition relaxation as important directions for future work.

We hope these clarifications and revisions address your concerns. Please let us know if there are any remaining questions or anything else we can improve that would help bring your evaluation to an accept.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank the four reviewers. The shared concerns were (i) comparisons on mismatched task sets and backbones, (ii) best-of-configuration reporting, (iii) no visible benefit of in-place repair, and (iv) dependence on a simulator. We re-ran the full evaluation to address (i)–(iii) and state (iv) as a limitation in the main text. Section, table and appendix numbers below refer to the revised PDF.

1. What changed

No.ConcernRaised byChange in the revisionWhere
1Different task sets and backbones per methodH898, VM3y, ves2All main comparisons use the same 62 tasks, the same backbone (Qwen3-VL-8B-Thinking), 5 seeds, 2 turns, and the same task inputs§5.1, Tables 2–4, 6
2Best-of selection over 56–57 configurationsves2Removed. One fixed configuration for every task§5.1, App. A.8, Table 8
3In-place repair shows no advantageH89824 new controlled tasks whose plan suffix contains free choices§5.5, Table 5
4Physics ablation on 26 runs, 1 seedves2310 runs (62 tasks × 5 seeds) and a new rehearse-only arm§5.5, Table 6
5Sufficient vs. required preconditionsH898New necessity (counter-example) test§4.2, App. A.4, Alg. 1 line 20
6Re-implemented baselines not validatedves2Each re-implementation reproduced on its original benchmark with its original modelsApp. E, Tables 15–17
7Seed-model construction unclearG91ZConstruction rule and a full worked example (true domain, seed domain, plan, failure, repair)App. F, Table 18
8NL-PDDL not discussedG91ZAdded to related work and to the comparison table§2, Table 1
9Library growth and redundancyves2New study from 38 to 102 predicates, with predicate retrievalApp. C.4, Tables 12–13
10Figures clutteredG91Z, VM3yFigures 2–4 redrawn around one running example eachFigs. 2–4
11Simulator dependence, no closed-loop real-robot testVM3y, ves2Stated as limitations in the main text; real-robot runs labelled qualitative§6, App. D, App. H

2. Main results under the matched protocol

OpenRepairBench, 62 tasks × 5 seeds, Qwen3-VL-8B-Thinking for every method, one configuration, no selection.

SettingMethodSuccessReuseD_sym (lower is better)VLM calls
Predicate invention (Table 2)SMOOTH93.5%76.8%1.94.6
LASP37.5%n/a (no memory)4.25.7
CLIMB24.3%37.3%3.87.3
Operator synthesis (Table 3)SMOOTH92.5%78.6%1.75.0
Lu et al.61.0%61.6%4.410.5
Compound (Table 4)SMOOTH89.3%62.1%1.67.1
VLM + Lu et al.46.5%40.6%5.215.6

Mini-BEHAVIOR (16 cases × 3 seeds, Table 2 bottom): SMOOTH 91.7% success with 3.9 calls; LASP 58.3% with 10.6; CLIMB 18.8% with 9.4.

Three notes on reading this table:

  • SMOOTH's numbers are lower than in the submission (100%, 100%, 95.8%). That is the effect of removing best-of selection and using one small open backbone. We consider the new numbers the correct ones.

  • Baselines receive the same task information as SMOOTH (deployment notes, initial scene, RGB failure image, action signatures) but no simulator access (§5.1). Testing hypotheses in simulation is the method. Its contribution is isolated inside SMOOTH in Table 6.

  • Lu et al. is run with a pre-formed skill in place of its RL skill learner, which is an upper bound. With its original learner it solves 9/62 tasks (App. E).

3. Ablations

Hypothesis testing (Table 6, 310 runs per arm):

ArmSuccessAccepted but invalidVLM calls / task
SMOOTH277/310 (89.3%)0.0%7.1
Rehearse-only (no hypothesis tests)203/310 (65.5%)0.0%14.3
No physics87/310 (28.1%)71.0%4.9

In-place splice vs. replanning with the same learned model (Table 5, 24 controlled tasks): D_sym 3.4 vs 6.6.

4. What the paper claims after the revision

  • Scope: to our knowledge, SMOOTH is the only framework that invents predicates and synthesizes operators online from an execution failure, and repairs the executing plan in place (Table 1).

  • Under a matched protocol it is ahead of each baseline by 31.5 to 69.2 points of success, with fewer VLM calls and lower plan disruption.

  • The gain comes from hypothesis testing, not from rehearsal alone (89.3% vs 65.5%).

5. Limitations we accept (§6)

  • SMOOTH needs a checkpointable simulator with readable scene variables. It is not shown to work without one.

  • No closed-loop repair on hardware. Repairs are computed in the twin and executed on the robot; the two pantry tasks are qualitative demonstrations (App. H).

  • New operators are chains of existing skills or compositions of given motion primitives. A repair that needs a fundamentally new continuous skill like “stirring” fails.

Dear Reviewers,

Thank you for your time and feedback. As the discussion period draws to a close, please take a moment to reply to the authors if you have yet to comment on their response or follow up on their latest reply.

Please let them know which concerns have been addressed and which, if any, remain. If you have no further questions, a brief confirmation is sufficient.

Best,
Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.