acceptodds
Under review as a conference paper at ICLR 2027

Code as Policies via Geometry-Conditioned Search

Abstract

Synthesizing robot-control programs from language requires geometric information that object poses alone do not provide, including functional directions and interaction locations. We introduce LOGIC, a geometry-conditioned policy synthesis framework that maps task instructions to a search space structured around geometric hypotheses and task-relevant control parameters. For each hypothesis, an LLM constructs an execution plan, identifies the parameters needed to carry it out, and generates parameterized control programs. Environment rollouts guide both parameter optimization and selection among hypothesis-conditioned programs. Execution feedback also refines the search space itself: failures guide revisions to the geometric hypotheses and task-relevant parameterizations that define subsequent policy candidates. New programs are then generated and optimized within this revised space. This combination of search-space construction, execution-based optimization, and feedback-driven refinement enables the synthesis of policies that compute end-effector poses and gripper commands without predefined task-level skill APIs. Experiments across manipulation and tool-use tasks, using both simulator-provided geometry and geometry estimated from RGB-D observations, show substantial improvements over code-as-policies and evolutionary baselines.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Code as policies, LLM program synthesis, Robot manipulation, Programmatic policies, Skill-free low-level control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4vt3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LOGIC, a framework for generating low-level robot control programs without predefined skill APIs. The method explicitly identifies geometric ambiguities, generates alternative hypotheses, and uses anticipatory planning to identify physical parameters for optimization. Candidate policies are optimized using TPE and refined through execution feedback. Experiments on RLBench, Robosuite, and LIBERO show substantial improvements over existing code-generation and evolutionary baselines.

Critical strengths
  1. The distinction between geometric semantic ambiguity and physical parameter ambiguity is well motivated. Exploring alternative geometric interpretations rather than committing to a single assumption seems particularly useful for low-level robotic control, where incorrect functional directions or interaction points can lead to systematic failures.
  2. LOGIC achieves 71% average success across 18 RLBench tasks, compared with 34% for the strongest baseline. Notably, LOGIC without evolution already achieves 65%, suggesting that the structured generation and optimization process contributes substantially. The RGB-D results are also encouraging, with 86% average success across 20 tasks.
Critical weaknesses
  1. LOGIC generates multiple programs for each geometric hypothesis and performs extensive parameter optimization. While the baselines also use TPE, it is unclear whether the overall search budgets are comparable. Could the authors compare against a baseline that generates a similar number of diverse programs under the same total LLM and environment-interaction budget, but without explicit geometric hypothesis branching? This would help clarify the contribution of the proposed search structure.

  2. While the overall framework is understandable, the paper does not clearly explain how execution feedback leads to changes in geometric hypotheses versus numerical parameters or control logic. The approach also depends on the LLM generating useful initial hypotheses and parameter ranges, but its robustness to incorrect or incomplete proposals is unclear. More concrete examples would make the method and its limitations easier to assess.

Questions for authors

Q1. Could the authors compare against a baseline that generates a similar number of diverse policies under matched LLM and environment-interaction budgets, but without explicit hypothesis branching? This would help isolate the contribution of the proposed geometric structure. Q2. The experiments use gpt-5-mini. Would the advantage of explicit hypothesis branching remain as large with a stronger reasoning model, such as GPT-6? Comparing LOGIC and the baselines under the same stronger backbone would help determine whether the framework addresses limitations that persist as model capabilities improve. Q3. When a policy fails, how does LOGIC distinguish between an incorrect geometric hypothesis, poor parameter values, and flawed control logic? Is this decision made entirely by the LLM? If a hypothesis changes, are the planning and parameter optimization stages repeated? Q4. The results contain many success-rate comparisons, but relatively little explanation of what actually changes during optimization and evolution. A step-by- step example showing an initial hypothesis, execution failure, and subsequent revision would be valuable. For tasks with lower success rates, such as PutUmbrellaInUmbrellaStand and OpenWashingMachine, what are the main failure causes? Q5. If the correct geometric interpretation is missing from the initial hypothesis space, or useful parameter values fall outside the LLM-proposed ranges, can the evolution process recover? This seems important for generalization to unfamiliar objects and more complex tasks.

6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/5)

We appreciate your insightful feedback and constructive comments! We address your comments point by point below (R1 to R6) and have revised the paper accordingly.

R1. Are the overall search budgets comparable? Could the authors compare against a baseline that generates a similar number of diverse programs under the same total LLM and environment-interaction budget, but without explicit geometric hypothesis branching? (Weakness 1, Q1)

Thank you for this suggestion. The search budgets are matched in the main comparison: every baseline in Table 1 receives the same total number of generated programs (86) and environment rollouts (172,000) as the full LOGIC pipeline (Appendix F.4). Both budgets the reviewer mentions, the LLM budget (LLM calls and LLM-generated programs) and the environment-interaction budget, are therefore matched between the full LOGIC pipeline and the baselines. Moreover, LOGIC without the evolution stage uses substantially fewer LLM calls and environment interactions than the baselines, and it still outperforms all of them (Table 1). In addition, the ablation "w/o Geometric Branching" in Appendix D.3 is the control you describe. It repeatedly samples diverse policies with anticipatory planning and explicit parameter generation, without assigning them to geometric branches, under the same overall search budget and the same per-policy TPE settings as the full method. In Table 3, this reduces success from 0.90 ± 0.03 to 0.53 ± 0.02 on OpenDrawer, from 0.74 ± 0.06 to 0.33 ± 0.06 on BlockPyramid, and from 0.76 ± 0.19 to 0.36 ± 0.07 on Hockey.

As this question is central to our contribution, we extended the component ablations to six more tasks, three with simulator-provided states and three under RGB-D perception, so that they now cover nine tasks. The first three rows are from the original submission (mean ± std over 5 seeds); the newly run ablations in the other six rows report mean ± std over 3 seeds, each evaluated on 50 held-out episodes. In these six rows, the LOGIC (Full) results, and the w/o Evolution results of the three simulator-state tasks, are those reported in the paper (5 seeds for the simulator-state tasks in Table 1, and 3 seeds for the RGB-D tasks in Table 2).

TypeTaskLOGIC (Full)w/o Evolutionw/o Physical Params Identificationw/o Geometric Branching
Oracle StateOpenDrawer0.90 ± 0.030.88 ± 0.040.77 ± 0.070.53 ± 0.02
Oracle StateBlockPyramid0.74 ± 0.060.62 ± 0.120.51 ± 0.020.33 ± 0.06
Oracle StateHockey0.76 ± 0.190.62 ± 0.230.25 ± 0.120.36 ± 0.07
Oracle StateTakeMoneyOutSafe0.67 ± 0.350.62 ± 0.340.64 ± 0.090.23 ± 0.33
Oracle StatePutItemInDrawer0.65 ± 0.230.57 ± 0.290.45 ± 0.210.20 ± 0.06
Oracle StateScoopWithSpatula0.83 ± 0.090.73 ± 0.100.70 ± 0.200.46 ± 0.13
Perceptionopen the microwave0.86 ± 0.030.82 ± 0.060.81 ± 0.030.57 ± 0.15
Perceptionstack the cube1.00 ± 0.001.00 ± 0.001.00 ± 0.000.98 ± 0.02
Perceptionturn on the stove0.93 ± 0.010.89 ± 0.120.82 ± 0.080.44 ± 0.24
Average (9 tasks)0.820.750.660.46

With the number of diverse programs and the interaction budget held fixed, removing explicit hypothesis branching lowers the average success rate from 0.82 to 0.46, the largest drop among all components. This shows that explicit hypothesis branching is the most important component of LOGIC, and that its advantage comes from the search structure and not from a larger budget. The nine-task ablation is now in the revised paper (Table 3).

AuthorsOpenReview ↗

Rebuttal by Authors (2/5)

R2. The experiments use gpt-5-mini. Would the advantage of explicit hypothesis branching remain as large with a stronger reasoning model, such as GPT-6? (Q2)

We studied the effect of model capability in Appendix D.5 (Table 6), where LOGIC is evaluated with gpt-5-mini, gpt-5, gpt-5.4-mini, and gpt-5.4. LOGIC scales with model capability (average success 0.80 with gpt-5-mini, 0.83 with gpt-5, and 0.82 with gpt-5.4), and the small gap between the standard and mini variants shows that the structured search lowers the dependence on raw model scale.

We agree that GPT-6 Astra, as the latest and strongest reasoning model, deserves a dedicated study, and that the baselines should be given the same backbone. We therefore ran LOGIC and all four baselines with GPT-6 Astra on six tasks. The GPT-6 Astra results are reported as mean ± std over 3 seeds, each evaluated on 50 held-out episodes. The LOGIC (GPT-5-mini) column gives the results reported in the paper (5 seeds for the simulator-state tasks in Table 1, and 3 seeds for the RGB-D tasks in Table 2).

TypeTaskLOGIC (GPT-5-mini)LOGIC (GPT-6 Astra)CaP (GPT-6 Astra)AlphaEvolve (GPT-6 Astra)EOH (GPT-6 Astra)MLES (GPT-6 Astra)
Oracle StateTakeMoneyOutSafe0.67 ± 0.350.70 ± 0.110.37 ± 0.080.40 ± 0.280.53 ± 0.060.22 ± 0.17
Oracle StatePutItemInDrawer0.65 ± 0.230.72 ± 0.030.44 ± 0.100.56 ± 0.140.52 ± 0.040.66 ± 0.18
Oracle StateScoopWithSpatula0.83 ± 0.090.88 ± 0.080.32 ± 0.120.45 ± 0.150.64 ± 0.060.42 ± 0.28
Perceptionopen the microwave0.86 ± 0.030.91 ± 0.080.82 ± 0.070.64 ± 0.090.70 ± 0.060.85 ± 0.14
Perceptionstack the cube1.00 ± 0.001.00 ± 0.000.98 ± 0.031.00 ± 0.001.00 ± 0.001.00 ± 0.00
Perceptionturn on the stove0.93 ± 0.010.97 ± 0.040.63 ± 0.020.92 ± 0.090.68 ± 0.110.64 ± 0.22
Average0.820.860.590.660.680.63

Three findings follow.

  • The advantage persists under the same stronger backbone. With GPT-6 Astra, LOGIC achieves the best result on every task, with an average of 0.86 compared with 0.59 to 0.68 for the baselines.
  • LOGIC with gpt-5-mini (0.82) still outperforms every baseline equipped with GPT-6 Astra (at most 0.68). The structure of the search contributes more than the upgrade of the backbone.
  • LOGIC itself continues to benefit from the stronger model (0.82 to 0.86), consistent with the trend in Table 6.

The reason is that the limitation LOGIC addresses is a lack of information, which stronger reasoning alone does not supply. Which local axis points toward a tool head, or on which side a hinge lies, is not determined by the task description and sparse observations, so even a strong model must commit to a guess unless alternatives are explored and tested by execution. This is consistent with Appendix F.2, where a strong VLM (GPT-5.4) asked to select a single interpretation from code, success rates, and three-camera renderings chose the correct one in only about one third of the cases (Table 10). The GPT-6 Astra results are now reported in Appendix D.5 (Table 7).

AuthorsOpenReview ↗

Rebuttal by Authors (3/5)

R3. When a policy fails, how does LOGIC distinguish between an incorrect geometric hypothesis, poor parameter values, and flawed control logic? Is this decision made entirely by the LLM? If a hypothesis changes, are the planning and parameter optimization stages repeated? (Weakness 2, Q3)

We thank the reviewer for the opportunity to clarify this. LOGIC addresses the three failure sources through complementary mechanisms. The LLM proposes diagnoses and revisions, while execution determines candidate selection.

  • Poor parameter values are addressed by TPE before evolution. Each candidate’s parameters are optimized under its current geometric hypothesis and program structure (Section 4.4). Failures that persist after this search motivate examining the hypothesis, control logic, missing parameters, and search ranges.
  • Incorrect geometric hypotheses are addressed through branching and execution-based selection. Candidate hypotheses are instantiated as separate branches and ranked by measured task success (Eq. 4). This deprioritizes poorly performing branches without requiring the LLM to determine which geometric interpretation is correct.
  • Residual failures are diagnosed in Stage 5. The LLM receives the policy code, the original prompt specifying the hypothesis, anticipatory plan, and parameter instructions, the measured success rate, and the final frame of a failed execution (Section 4.5, Appendix I.5). These explicit assumptions and control steps provide a basis for identifying a likely cause, such as an incorrect opening arc (Appendix E.2) or a collision with the stick head before grasping in Hockey (Section 4.5).

Each proposed revision undergoes parameter re-optimization with the full TPE budget and joins the candidate pool. The elite set is then re-selected from parents and revised candidates by measured success rate (Algorithm 1). Revisions are therefore evaluated by their execution performance rather than accepted solely on the LLM’s diagnosis.

When a hypothesis changes, the LLM jointly revises the affected plan steps, waypoints, parameters, and search ranges within a single evolution step, followed by full parameter re-optimization. If the interpretation was absent from the initial hypotheses, the revised program enters the pool as a new branch. We describe this procedure in Section 4.5 and give two complete revision examples in Appendix F.6 of the revised paper.

Reviewer_33S1OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LOGIC, a geometry-conditioned framework for synthesizing programmatic policies for robotic manipulation. Without relying on predefined task-level skill APIs, the method uses a large language model to propose task-relevant geometric hypotheses, identify continuous control parameters through anticipatory planning, and generate programs that directly compute target end-effector poses and gripper commands. The framework then uses environment rollouts to optimize parameters and select candidates, while failure feedback guides revisions to geometric hypotheses, parameterizations, and control logic. The authors evaluate LOGIC on simulated tasks from RLBench, Robosuite, and LIBERO using both simulator-provided states and RGB-D-derived geometric information. The experiments report improved success rates over code-generation and evolutionary-search baselines, with ablations further supporting the contributions of explicit geometric branching and planning-guided parameter identification.

Critical strengths
  1. The method addresses a well-motivated problem. Separating geometric hypotheses from continuous parameter optimization directly targets an important difficulty in low-level code-based robot control.
  2. The reported improvements are substantial. LOGIC achieves 71% average success on RLBench versus 34% for the strongest evaluated baseline. Without evolution, it still reaches 65%. The appendix states that the main comparisons match program and rollout budgets.
  3. The ablations support the central mechanism. Removing geometric branching substantially reduces performance on all three representative tasks, supporting the value of explicitly structured search.
Critical weaknesses
  1. The advantage over stronger parameterized baselines remains insufficiently isolated. Baselines receive TPE optimization through regex-based extraction of numerical constants, which may produce poorly structured search spaces compared with LOGIC’s semantically meaningful parameters. Although the component ablations help, they cover only three tasks. A stronger baseline using explicit parameter generation without geometric branching across the full benchmark would better establish the source and generality of the gains.

  2. High synthesis cost limits practical applicability. Appendix F.4 reports approximately 172,000 rollouts and 58 hours per typical task. This dependence on extensive environment access substantially limits deployment when resets are costly or unsafe. All experiments—including the RGB-D evaluation—remain in simulation, leaving physical execution reliability and sim-to-real transfer untested.

Questions for authors

Could you please address the concerns raised in the Weaknesses section?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

We appreciate your insightful feedback and constructive comments! We address your comments point by point below (R1 and R2) and have revised the paper accordingly.

R1. The advantage over stronger parameterized baselines is insufficiently isolated: regex-based extraction may give the baselines poorly structured search spaces, and the component ablations cover only three tasks. A baseline using explicit parameter generation without geometric branching would better establish the source and generality of the gains.

We thank the reviewer for this valuable suggestion. The ablation "w/o Geometric Branching" in Appendix D.3 was designed for this purpose: it keeps anticipatory planning and explicit parameter generation for every sampled policy, removes only the hypothesis branches, and uses the same overall search budget. To address your concern more thoroughly, we conducted two additional experiments.

(1) Baselines with explicit parameter generation. We re-ran all four baselines so that the LLM itself generates explicit, semantically meaningful parameters together with diverse programs, instead of relying on regex-based extraction of numerical constants. These baselines therefore have a well-structured parameter space but no explicit geometric branching. As in the main comparison, every baseline receives the same total number of generated programs and environment rollouts as LOGIC. We evaluated six tasks, three with simulator-provided states and three under RGB-D perception. The baseline results are reported as mean ± std over 3 seeds, each evaluated on 50 held-out episodes. The LOGIC results are those reported in the paper (5 seeds for the simulator-state tasks in Table 1, and 3 seeds for the RGB-D tasks in Table 2).

TypeTaskLOGICCaPAlphaEvolveEOHMLES
Oracle StateTakeMoneyOutSafe0.67 ± 0.350.16 ± 0.170.13 ± 0.190.24 ± 0.330.20 ± 0.06
Oracle StatePutItemInDrawer0.65 ± 0.230.21 ± 0.160.26 ± 0.240.45 ± 0.140.24 ± 0.14
Oracle StateScoopWithSpatula0.83 ± 0.090.30 ± 0.270.49 ± 0.170.46 ± 0.130.62 ± 0.13
Perceptionopen the microwave0.86 ± 0.030.42 ± 0.160.56 ± 0.200.60 ± 0.140.73 ± 0.12
Perceptionstack the cube1.00 ± 0.000.94 ± 0.020.98 ± 0.030.98 ± 0.030.98 ± 0.02
Perceptionturn on the stove0.93 ± 0.010.58 ± 0.160.87 ± 0.040.73 ± 0.120.84 ± 0.03
Average0.820.440.550.580.60

LOGIC achieves the highest success rate on all six tasks, with an average of 0.82 compared with 0.44 to 0.60 for the explicitly parameterized baselines. Explicit parameters do help the baselines compared with regex extraction (e.g., on PutItemInDrawer, EOH improves from 0.12 ± 0.15 in Table 1 to 0.45 ± 0.14), which confirms the value of semantically meaningful parameters that LOGIC identifies through anticipatory planning. However, a clear gap remains on tasks where the functional direction or interaction location is ambiguous (e.g., TakeMoneyOutSafe 0.67 vs. at most 0.24). This isolates explicit geometric branching as a source of the gain.

(2) Component ablations on six more tasks. We appreciate your point that the component ablations should cover more tasks, and we extended them from three to nine tasks, with the results below. New ablations use 3 seeds (50 held-out episodes each); other results are from the paper.

TypeTaskLOGIC (Full)w/o Evolutionw/o Physical Params Identificationw/o Geometric Branching
Oracle StateOpenDrawer0.90 ± 0.030.88 ± 0.040.77 ± 0.070.53 ± 0.02
Oracle StateBlockPyramid0.74 ± 0.060.62 ± 0.120.51 ± 0.020.33 ± 0.06
Oracle StateHockey0.76 ± 0.190.62 ± 0.230.25 ± 0.120.36 ± 0.07
Oracle StateTakeMoneyOutSafe0.67 ± 0.350.62 ± 0.340.64 ± 0.090.23 ± 0.33
Oracle StatePutItemInDrawer0.65 ± 0.230.57 ± 0.290.45 ± 0.210.20 ± 0.06
Oracle StateScoopWithSpatula0.83 ± 0.090.73 ± 0.100.70 ± 0.200.46 ± 0.13
Perceptionopen the microwave0.86 ± 0.030.82 ± 0.060.81 ± 0.030.57 ± 0.15
Perceptionstack the cube1.00 ± 0.001.00 ± 0.001.00 ± 0.000.98 ± 0.02
Perceptionturn on the stove0.93 ± 0.010.89 ± 0.120.82 ± 0.080.44 ± 0.24
Average (9 tasks)0.820.750.660.46

The conclusions of the original ablation still hold. Geometric branching remains the most important component, and removing it causes the largest drop under the same search budget. Branching ensures that the correct interpretation is always tested, whereas without it the budget is often spent tuning programs built on a wrong one. Both experiments are now in the revised paper (Tables 3 and 5).

AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

R2. High synthesis cost limits practical applicability (about 172,000 rollouts and 58 hours per task), and all experiments, including the RGB-D evaluation, remain in simulation.

Thank you for raising this point. We would like to clarify the cost from three aspects.

(1) The cost is matched across methods, and LOGIC uses it most efficiently. Every baseline in Table 1 receives the same number of generated programs (86) and environment rollouts (172,000) as LOGIC (Appendix F.4). Under this equal budget, LOGIC reaches 0.71 average success compared with 0.27 to 0.34 for the baselines. In wall-clock time, LOGIC takes 58 hours, while the evolutionary baselines take 238 to 277 hours for the same program budget (Table 13), because hypothesis-conditioned programs can be generated and optimized in parallel.

(2) Most of the gain is obtained with a fraction of the budget. LOGIC without evolution uses 36 of the 86 programs and 72,000 of the 172,000 rollouts, finishes in about 10 hours, and already reaches 0.65 average success. Figure 4 further shows on three representative tasks that LOGIC converges faster and reaches a higher success rate than all baselines, both in terms of environment resets and within a 10-hour wall-clock window.

(3) The cost is a one-time, automated, offline cost. The search runs in simulation, where rollouts, resets, and success signals are inexpensive and safe, and it requires only about 93 LLM calls per task (Table 12). Its output is a reusable control program that is then executed without any further LLM call or search, so the synthesis cost is amortized over all later executions. The costly or unsafe resets mentioned by the reviewer are therefore not required at deployment, since only the synthesized program is executed.

Regarding physical execution, we designed the evaluation to cover the conditions that matter for deployment:

  • Observation interface. The RGB-D setting (Section 5.3, Appendix E) removes simulator-provided object poses. The policy only receives images, camera calibration, and robot state, and geometry is estimated by SAM3 segmentation, depth back-projection, and OBB fitting. Under this interface LOGIC reaches 0.86 average success over 20 tasks from RLBench, Robosuite, and LIBERO.
  • Observation noise. With position noise (σ = 1 cm), orientation noise (σ = 5°), and detection dropout (p = 0.05), LOGIC remains well above CaP on all three tasks (Table 9), e.g., 0.78 ± 0.08 vs. 0.14 ± 0.10 on OpenDrawer.
  • Action interface. The program outputs an SE(3) end-effector target and a gripper command, the standard interface of real robot stacks, where a mature component (e.g., MoveIt) handles inverse kinematics and motion execution (Appendix A.1). The same pipeline runs on RLBench, Robosuite (OSC control, Appendix F.3), and LIBERO.
  • Inspectability. The synthesized program exposes its geometric assumption and a small set of physically meaningful parameters (e.g., grasp offsets and clearances in meters), which gives a direct handle for checking and adjusting a controller before it is run on hardware.

Validating the synthesized programs on a physical robot is the next step of this work, as discussed in Section 7 and Appendix C of the revised paper.

The revised paper reflects all of the above results and discussions.

AuthorsOpenReview ↗

Author follow-up: summary of our response

Dear Reviewer 33S1,

Thank you for your thoughtful review. In our response we addressed your main concerns:

  • Baselines with explicit parameter generation. We re-ran all baselines with LLM-generated, semantically meaningful parameters instead of regex extraction, giving them a well-structured parameter space but no geometric branching. LOGIC achieves the best result on all six tasks (0.82 vs. at most 0.60). Explicit parameters help the baselines, which confirms the value of LOGIC's parameter identification, but a clear gap remains where the functional direction or interaction location is ambiguous, isolating geometric branching as a key source of the gain.
  • Ablation experiments on more tasks. We extended the component ablations to six more tasks, covering both oracle-state and RGB-D settings. The original conclusions still hold: geometric branching remains the most important component, and removing it lowers average success from 0.82 to 0.46.
  • Synthesis cost. Under the same budget as all baselines, LOGIC reaches 0.71 vs. at most 0.34 and runs several times faster than the evolutionary baselines. LOGIC without evolution uses less than half of the budget and already outperforms all of them. The cost is a one-time offline cost, and the synthesized program can be reused without further search, so no costly resets are needed at deployment.
  • Beyond simulation. We designed our evaluation to cover the conditions that matter for physical deployment. LOGIC works from RGB-D images without simulator poses and reaches 0.86 over 20 tasks across three benchmarks. It also stays robust under observation noise and outputs end-effector poses, the standard interface of real robot stacks.

Both new experiments are now in the revised paper. We would be grateful to hear whether these additions resolve your concerns and support re-evaluation. We sincerely appreciate your time.

Thank you for your response. I have decided to maintain my original score.

Reviewer_GM5SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LOGIC, a framework for construct, search, and refine a structured space of robotic control programs. It constructs geometric hypotheses, uses anticipatory planning to identify control parameters, and generates parameterized programs that output end-effector targets and gripper commands. Environment rollouts guide TPE parameter optimization and candidate selection, followed by feedback-driven refinement of the hypotheses and programs. Experiments cover simulator-state and RGB-D observation settings in RLBench, Robosuite, and LIBERO.

Critical strengths
  1. Structuring program search around explicit geometric hypotheses and task-relevant parameters is interesting. Ablation study demonstrates removing framework components would cause performance drop.

  2. The reported results show great improvements over the evaluated baselines. LOGIC reports 71% mean success on 18 RLBench tasks, compared with 27–34% for the baselines. The 20-task RGB-D evaluation provide further evidence for the approach.

Critical weaknesses
  1. The synthesis cost is substantial. Table 12 reports approximately 172,000 rollouts and 58 hours per task. The search multiplies hypothesis branches, candidate programs, and parameter trials, requiring extensive environment interaction and resets.

  2. Practical applicability is insufficiently demonstrated. All experiments are in simulation, with no real-robot validation or deployment-time latency analysis. Applying the proposed search directly on a physical robot would require a prohibitive number of executions and scene resets, together with a reliable and efficient way to determine task success. The paper does not establish how these requirements would be met in practice or validate transferring a simulator-synthesized controller to hardware. Even without evolution, synthesis takes approximately 10 hours per task, leaving efficient adaptation to new physical tasks unproven.

  3. The comparison with alternative approaches remains limited. In the RGB-D evaluation, LOGIC uses approximately 2.7M environment steps versus 0.25M for the main CaP-Agent0 baseline. The additional interaction-matched controls are useful but cover only five of twenty tasks (Tables 7–8). There is also no empirical comparison with classic VLA policy, leaving the performance gain against learned robot policies unclear.

Questions for authors
  1. Are the 50-episode evaluations used for evolution ranking a separate validation set? Please explain the detailed evaluation setting used for Table 1, including its initialization split and the point at which the program is frozen.

  2. Please provide the rollout budgets, run configurations, and candidate-selection procedures for each baseline. How will the baseline performs when giving the same level of computation budgets? Can the Cap-Agent0 baseline comparison be extended beyond the five-task subset?

  3. Can you provide a task-level comparison with some representative VLA policy? Is it possible to deploy the proposed method to real physical robot?

7 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/4)

We appreciate your insightful feedback and constructive comments! We address your comments point by point below (R1 to R6) and have revised the paper accordingly.

R1. The synthesis cost is substantial (about 172,000 rollouts and 58 hours per task).

We thank the reviewer for raising this point. We would like to clarify three aspects of the cost reported in Table 12.

(1) The comparison in Table 1 is budget-matched. Every baseline (CaP, AlphaEvolve, EOH, MLES) receives the same number of generated programs and environment rollouts as the full LOGIC pipeline (Appendix F.4). Under this equal budget, LOGIC is also the most efficient search procedure in wall-clock time, because hypothesis-conditioned programs are generated and optimized in parallel. The table below consolidates Tables 1, 12, and 13 of the paper.

MethodGenerated programsEnvironment rolloutsWall-clock (hours)Average SR (18 RLBench tasks)
CaP86172,000100.27
AlphaEvolve86172,0002380.34
EOH86172,0002770.31
MLES86172,0002460.30
LOGIC (w/o Evol.)3672,000100.65
LOGIC (Full)86172,000580.71

(2) Most of the gain is obtained early. LOGIC without the evolution stage uses 36 of the 86 programs and 72,000 of the 172,000 rollouts, finishes in about 10 hours, and already reaches 0.65 average success, well above every baseline given the full budget (0.27 to 0.34). The learning curves in Figure 4 (Appendix D.2) show the same trend on three representative tasks: LOGIC converges faster and reaches a higher success rate than all baselines, both in terms of environment resets and within a 10-hour wall-clock window.

(3) The cost is a one-time, fully automated, offline cost paid in simulation, and it is light on LLM usage. A typical task needs only about 93 LLM calls (Table 12). In the RGB-D setting LOGIC uses about 29 LLM requests and 0.22M tokens per task, which is 19x fewer requests and 31x fewer tokens than CaP-Agent0 (Table 8). The result of the synthesis is a reusable control program: once synthesized, it is executed on new initializations without any further LLM call or search, so the cost is amortized over all subsequent executions.

This budget comparison now appears in the main paper (Table 1 and Section 5.2).

R2. Practical applicability: no real-robot validation or deployment-time latency analysis; the search cannot be run directly on a physical robot. Is it possible to deploy the method on a real robot?

LOGIC targets offline policy synthesis in a simulator, followed by reuse of the resulting control program. The synthesis rollouts, resets, and success evaluations therefore take place in simulation. At deployment, the frozen program would compute end-effector targets and gripper commands from perceived geometry and robot state, without further LLM calls or program search. This deployment scenario assumes access to a suitable task simulator.

Our RGB-D and observation-noise experiments provide evidence that LOGIC can operate with estimated geometry and imperfect observations:

  • Observation interface. The RGB-D setting (Section 5.3, Appendix E) removes simulator-provided object poses. The policy only receives images, camera calibration, and robot state, and geometry is estimated by SAM3 segmentation, depth back-projection, and OBB fitting. Under this interface LOGIC reaches 0.86 average success over 20 tasks from RLBench, Robosuite, and LIBERO.
  • Observation noise. With position noise (σ = 1 cm), orientation noise (σ = 5°), and detection dropout (p = 0.05), LOGIC remains well above CaP on all three tasks (Table 9), e.g., 0.78 ± 0.08 vs. 0.14 ± 0.10 on OpenDrawer.
  • Action interface. The program outputs an SE(3) end-effector target and a gripper command, the standard interface of real robot stacks, where a mature component (e.g., MoveIt) handles inverse kinematics and motion execution (Appendix A.1). The same pipeline runs on RLBench, Robosuite (OSC control, Appendix F.3), and LIBERO.
  • Inspectability. The synthesized program exposes its geometric assumption and a small set of physically meaningful parameters (e.g., grasp offsets and clearances in meters), which gives a direct handle for checking and adjusting a controller before it is run on hardware.

However, we agree that these results do not directly establish reliable transfer to physical robots. The revised paper states this distinction in Section 7 and Appendix C and leaves deployment on physical hardware to future work.

AuthorsOpenReview ↗

Rebuttal by Authors (2/4)

R3. The additional interaction-matched Agent0 controls cover only five of twenty tasks. Can the CaP-Agent0 comparison be extended beyond the five-task subset?

Thank you for the suggestion. Following your advice, we extended both controls to all 20 RGB-D tasks. Agent0 + TPE applies the same TPE routine to the policy produced after 10 feedback rounds, and Agent0-100feedbacks raises the feedback limit from 10 to 100 rounds. Both controls use about the same number of environment steps as LOGIC (around 2.7M per task). Results are reported as mean ± std over 3 seeds, each evaluated on 50 held-out episodes.

BenchmarkTaskLOGICAgent0Agent0 + TPEAgent0-100feedbacks
RLBenchopen the microwave0.86 ± 0.030.04 ± 0.040.60 ± 0.030.53 ± 0.20
RLBenchopen the drawer0.85 ± 0.050.14 ± 0.020.42 ± 0.060.71 ± 0.07
RLBenchslide the cabinet door open0.55 ± 0.180.22 ± 0.080.43 ± 0.100.40 ± 0.03
RLBenchclose the fridge0.86 ± 0.070.73 ± 0.020.83 ± 0.060.89 ± 0.02
RLBenchpush the button0.98 ± 0.020.87 ± 0.020.94 ± 0.060.92 ± 0.02
RLBenchclose the drawer0.84 ± 0.060.86 ± 0.030.89 ± 0.040.92 ± 0.03
RLBenchclose the laptop lid0.56 ± 0.020.73 ± 0.090.75 ± 0.100.84 ± 0.11
Robosuitestack the cube1.00 ± 0.000.90 ± 0.021.00 ± 0.000.97 ± 0.02
Robosuitelift the cube0.99 ± 0.010.78 ± 0.030.83 ± 0.110.79 ± 0.04
LIBEROturn on the stove0.93 ± 0.010.11 ± 0.060.72 ± 0.150.48 ± 0.10
LIBEROopen the top drawer of the cabinet0.98 ± 0.020.03 ± 0.020.25 ± 0.030.04 ± 0.04
LIBEROturn off the stove0.45 ± 0.010.17 ± 0.050.27 ± 0.090.24 ± 0.20
LIBEROput the bowl on top of the cabinet1.00 ± 0.000.95 ± 0.020.96 ± 0.061.00 ± 0.00
LIBEROput the bowl on the plate0.95 ± 0.030.30 ± 0.150.38 ± 0.050.49 ± 0.14
LIBEROput the tomato sauce in the basket1.00 ± 0.000.43 ± 0.040.96 ± 0.020.82 ± 0.05
LIBEROput the orange juice in the basket1.00 ± 0.000.51 ± 0.190.72 ± 0.040.70 ± 0.19
LIBEROput the wine bottle on top of the cabinet1.00 ± 0.000.19 ± 0.190.91 ± 0.100.50 ± 0.24
LIBEROput the red mug on the plate0.92 ± 0.020.35 ± 0.010.68 ± 0.080.42 ± 0.24
LIBEROopen the microwave0.97 ± 0.010.02 ± 0.020.46 ± 0.240.26 ± 0.14
LIBEROput the frying pan on the stove0.54 ± 0.200.48 ± 0.060.51 ± 0.070.52 ± 0.12
Average (20 tasks)0.860.440.680.62

With a matched interaction budget, LOGIC keeps the highest average success rate (0.86 vs. 0.68 and 0.62) and achieves the best result on 17 of the 20 tasks. The advantage is largest on articulated-object tasks, where the estimated object center does not specify the contact location or the opening motion: 0.98 vs. 0.25 / 0.04 on LIBERO "open the top drawer of the cabinet", 0.97 vs. 0.46 / 0.26 on LIBERO "open the microwave", and 0.86 vs. 0.60 / 0.53 on RLBench "open the microwave". This shows that additional environment interaction alone does not close the gap; the gain comes from spending the interaction on a search space organized by geometric hypotheses. LOGIC also achieves this with far fewer LLM resources: about 29 requests and 0.22M tokens per task, compared with about 560 requests and 6.9M tokens for Agent0 + TPE and about 5.6K requests and 69M tokens for Agent0-100feedbacks (Table 8). The full 20-task comparison is now reported in Table 2 of the revised paper.

AuthorsOpenReview ↗

Rebuttal by Authors (3/4)

R4. There is no empirical comparison with a VLA policy. Can you provide a task-level comparison with some representative VLA policies?

We appreciate this suggestion. The initial submission compared with VLA models conceptually (Appendix A.1, Table 4). We have now added a task-level empirical comparison with three representative VLA policies on all 20 RGB-D tasks, using their publicly released checkpoints without additional training: OpenVLA-7B (openvla/openvla-7b-finetuned-libero-goal), π0 (pi0_libero), and π0.5 (pi05_libero). Results are reported as mean ± std over 3 seeds, each evaluated on 50 held-out episodes.

BenchmarkTaskLOGICOpenVLA-7Bπ0π0.5
RLBenchopen the microwave0.86 ± 0.030.00 ± 0.000.00 ± 0.000.04 ± 0.06
RLBenchopen the drawer0.85 ± 0.050.00 ± 0.000.00 ± 0.000.00 ± 0.00
RLBenchslide the cabinet door open0.55 ± 0.180.04 ± 0.060.00 ± 0.000.00 ± 0.00
RLBenchclose the fridge0.86 ± 0.070.10 ± 0.140.00 ± 0.000.34 ± 0.14
RLBenchpush the button0.98 ± 0.020.07 ± 0.100.00 ± 0.000.51 ± 0.11
RLBenchclose the drawer0.84 ± 0.060.00 ± 0.000.00 ± 0.000.23 ± 0.32
RLBenchclose the laptop lid0.56 ± 0.020.08 ± 0.110.00 ± 0.000.04 ± 0.06
Robosuitestack the cube1.00 ± 0.000.00 ± 0.000.00 ± 0.000.00 ± 0.00
Robosuitelift the cube0.99 ± 0.010.00 ± 0.000.00 ± 0.000.00 ± 0.00
LIBEROturn on the stove0.93 ± 0.010.92 ± 0.031.00 ± 0.001.00 ± 0.00
LIBEROopen the top drawer of the cabinet0.98 ± 0.020.00 ± 0.000.00 ± 0.000.00 ± 0.00
LIBEROturn off the stove0.45 ± 0.010.00 ± 0.000.00 ± 0.000.00 ± 0.00
LIBEROput the bowl on top of the cabinet1.00 ± 0.001.00 ± 0.000.95 ± 0.040.98 ± 0.03
LIBEROput the bowl on the plate0.95 ± 0.030.87 ± 0.061.00 ± 0.001.00 ± 0.00
LIBEROput the tomato sauce in the basket1.00 ± 0.000.56 ± 0.140.96 ± 0.040.98 ± 0.02
LIBEROput the orange juice in the basket1.00 ± 0.000.94 ± 0.030.93 ± 0.090.96 ± 0.02
LIBEROput the wine bottle on top of the cabinet1.00 ± 0.000.80 ± 0.160.96 ± 0.040.94 ± 0.05
LIBEROput the red mug on the plate0.92 ± 0.020.00 ± 0.000.13 ± 0.070.46 ± 0.11
LIBEROopen the microwave0.97 ± 0.010.00 ± 0.000.00 ± 0.000.00 ± 0.00
LIBEROput the frying pan on the stove0.54 ± 0.200.00 ± 0.000.00 ± 0.000.00 ± 0.00
Average (20 tasks)0.860.270.300.37

LOGIC achieves the highest average success rate (0.86 vs. 0.27, 0.30, and 0.37) and the best result on 18 of the 20 tasks. Two observations follow from the task-level results.

  • On LIBERO tasks that are well covered by the demonstrations used to fine-tune these checkpoints (e.g., placing the bowl, tomato sauce, orange juice, and wine bottle), the VLA policies are strong, and LOGIC reaches the same level (0.95 to 1.00) without using any demonstration.
  • On tasks outside that coverage, LOGIC remains effective while the VLA policies do not transfer. This includes LIBERO tasks that require reasoning about articulation and contact locations ("open the top drawer of the cabinet" 0.98 vs. 0.00, "open the microwave" 0.97 vs. 0.00), and the RLBench and Robosuite tasks, which present a new scene and embodiment to these checkpoints. Adapting a VLA to such tasks requires collecting task-specific demonstrations, whereas LOGIC synthesizes a controller for each of them from the task description and execution rollouts alone.

These results clarify the performance gain against learned robot policies and show that LOGIC is a complementary, demonstration-free route. The VLA comparison is now part of Table 2 in the revised paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Global comment for all reviewers

We sincerely thank all reviewers for their valuable feedback and constructive suggestions! We have responded to each reviewer individually and uploaded a revised paper. Changes in the main text are marked in blue, and changes in the appendix are marked in red. The main revisions are summarized below:

(1) Search budget [Reviewers GM5S, 33S1, 4vt3]. Table 1 now reports the generated programs, environment rollouts, and wall-clock hours of every method, and Section 5.2 discusses this comparison.

(2) Comparisons under RGB-D perception [Reviewer GM5S]. Table 2 now reports the two interaction-matched CaP-Agent0 controls on all 20 tasks and adds a comparison with VLA policies (Section 5.3 and Appendix E.3).

(3) Ablations and stronger baselines [Reviewers 33S1, 4vt3]. The component ablations are extended from three to nine tasks (Table 3), and baselines with explicit parameter identification are added (Table 5 in Appendix D.4).

(4) Stronger backbone model [Reviewer 4vt3]. Results of LOGIC and all baselines with GPT-6 Astra are added (Table 7 in Appendix D.5).

(5) Evaluation protocol [Reviewer GM5S]. Section 5.1 and Appendix F.1 now state the held-out evaluation protocol and the point at which the program is frozen.

(6) Evolution examples and failure analysis [Reviewer 4vt3]. Appendix F.6 adds two complete revision examples and a failure analysis of the two tasks with the lowest success rates.

(7) Real-robot deployment [Reviewers GM5S, 33S1]. Section 7 and Appendix C discuss the simulation-only evaluation and leave deployment on physical hardware to future work.

We thank the reviewers again for helping us improve the paper, and we are glad to answer any further questions.

AuthorsOpenReview ↗

Author Final Remarks

We sincerely thank all reviewers for their valuable comments and for recognizing the contribution of our work. We summarize below the main concerns raised and how each is addressed in our responses.

(1) Clarification of search budget [Reviewer GM5S, 4vt3]: All baselines in Table 1 are already evaluated under the same budget as LOGIC, with 86 generated programs and 172,000 rollouts per task, the same action interface, the same parameter tuner, and the same disjoint optimization and held-out splits. LOGIC reaches 0.71 average success against 0.27 to 0.34, and even without evolution it reaches 0.65 using less than half of that budget. The advantage therefore comes from structuring the search around explicit geometric hypotheses, not from larger search budgets.

(2) Source of the gain [Reviewer 33S1, 4vt3]: To rule out better parameterization as the explanation, we re-ran all four baselines with LLM-generated explicit parameters on six tasks, which gives them a well-structured parameter space but no geometric branching, and we extended the component ablations from three to nine tasks. LOGIC outperforms all baselines equipped with explicit parameters on all six tasks (0.82 vs. at most 0.60), while removing branching alone lowers average success from 0.82 to 0.46 under a fixed budget. This isolates explicit hypothesis branching as the source of the improvement.

(3) Stronger backbone model [Reviewer 4vt3]: A stronger backbone does not change the conclusion. When equipping all methods with GPT-6 Astra, LOGIC achieves the best result on every task (0.86 vs. 0.59 to 0.68), and LOGIC with gpt-5-mini (0.82) still outperforms every baseline using GPT-6 Astra. The missing geometry is simply not contained in the task description, so a stronger model cannot supply it and the structured search matters more than the backbone.

(4) Comparison with CaP-Agent0 and VLA policies [Reviewer GM5S]: Following the reviewer's suggestion, we extended both interaction-matched CaP-Agent0 variants from five to all 20 RGB-D tasks and added a task-level comparison with three representative VLA policies. LOGIC reaches 0.86 average success against 0.68 and 0.62 for the interaction-matched Agent0, and against at most 0.37 for the VLA policies, with far fewer LLM requests and without using any demonstrations. More interaction or demonstration pretraining therefore does not substitute for resolving the geometric ambiguity itself.

(5) Failure attribution and the evolution examples [Reviewer 4vt3]: Failures are never attributed by the LLM alone, since poor parameters are absorbed by TPE, wrong hypotheses are deprioritized by execution-based selection, and each LLM revision is re-optimized and retained only if measured success improves. We also traced two complete policy revisions, one recovering a correct interpretation that had been removed from the initial hypothesis set (held-out success 0.48 to 0.72), and one retaining the hypothesis while rewriting the control logic and the parameter ranges (0.28 to 0.46).

(6) Synthesis cost and real-robot deployment [Reviewer GM5S, 33S1]: Synthesis incurs a one-time offline cost, and LOGIC is faster than the evolutionary baselines under matched budgets. The resulting program is reused without further LLM calls or search. Our contribution addresses geometric ambiguity in translating language and sparse observations into control programs, a problem present in both simulated and physical environments. Simulation provides a controlled setting for evaluating this synthesis mechanism, consistent with established practice in related work (e.g., RoboGen, ICML 2024; Voyager, TMLR 2024; MLES, ICLR 2026). Our RGB-D evaluation excludes ground-truth object poses and articulation states and adopts the perception and control interface of CaP-X, which has demonstrated code-based manipulation on physical robots. LOGIC therefore builds on an interface with demonstrated real-robot applicability.

All of the additional results and discussion are incorporated into the revised paper. We thank all reviewers again for their valuable comments, which have helped us improve this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.