acceptodds
Under review as a conference paper at ICLR 2027

Agentic Robot Creation through Generation, Verification, and Revision

Abstract

Can an AI agent turn high-level user intent into a functioning robot and teach it how to behave? We study this question from two complementary aspects: embodiment and behavior creation. For embodiment creation, we investigate how well current multimodal models capture mechanical requirements, and introduce computational metrics for evaluating generated designs before fabrication. For behavior creation, we present RoboForge, an agentic framework that turns task specifications into executable policies through motion generation, policy learning, and execution-grounded revision. Given a robot embodiment and task, the agent generates a reference motion, configures the learning objective, and trains a policy. Crucially, motion remains in the loop: rollout feedback can revise the reference motion, reward, or both. We evaluate a generated embodiment across multiple MLLMs and behavior creation across four diverse robots against reward-refinement and one-shot motion baselines. Our computational metrics expose recurring mechanical failures in generated designs, while behavior experiments show that execution-grounded refinement improves task-level behavior and execution quality. Finally, we fabricate an agent-generated Gorilla and directly deploy learned behaviors on the physical robot.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Agentic Robot Creation, Robot Behavior Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zYmROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies agentic robot creation through two complementary components. First, it evaluates MLLM-generated robot embodiments with diverse checks covering assembly, mechanical feasibility, and motion execution. Second, it introduces RoboForge, a framework that generates a reference motion and reward, trains a policy, inspects its execution, and revises the motion, reward, or both. Experiments cover 19 tasks across four embodiments. RoboForge achieves a better performance and mean human rating than other methods.

Critical strengths
  1. The proposed feedback loop is well motivated. RoboForge bridges the gap between motion generation and executable policy learning by physical execution result. Diagnosing failures from rollouts to jointly revise both the reference motion and the optimization reward helps the system to discover more learnable behaviors.

  2. The evaluation is comprehensive and multi-faceted. The authors show the performance across 19 diverse tasks and effectively pair qualitative user study with quantitative, reference-independent physical metrics, like yaw error, foot-slip, action variation.

Critical weaknesses
  1. The experiments do not isolate the claimed benefit of motion refinement. The comparison measures the benefit of the entire additional search-and-revision process, not specifically motion refinement. More importantly, the paper lacks a fixed-reference, reward-refinement baseline using the same motion-tracking scaffold and budget. Eureka-style has no reference motion, so it cannot provide this control.
  2. Although the experiments cover 19 tasks, they involve only four robot embodiments. There remains limited support for generalization to substantially different morphologies and mechanical configurations. Also, section 3.2 compares only one gorilla-inspired design from each of four models.
  3. Insufficient detail on human intervention. As section 3.2 states, the remaining design issues were manually corrected before fabrication, but the procedures are not adequately documented. Since the strongest generated design passes only 15/23 checks in Table 1, with remaining failures involving several issues. The authors are encouraged to show how these things are validated and changed.
Questions for authors

See weakness.

3 comments

We thank the reviewer for recognizing the value of our feedback loop and the breadth of our evaluation. We focus on isolating the contribution of motion revision, clarifying morphological coverage, reporting repeated generation across model families, and documenting human intervention before fabrication.

W1. Does motion revision add value beyond reward-only refinement?

Original weakness: The experiments do not isolate the claimed benefit of motion refinement.

Thank you for the suggestion. We added a 2×2\boldsymbol{2 \times 2} ablation that independently enables or disables motion revision and reward revision (Tables R3–R4). Compared with Only Reward, which fixes the reference motion, Ours achieves better means on all four physical metrics and a higher mean user rating. This supports the additional value of allowing motion revision while retaining reward revision. We have added the ablation settings, physical metrics, and overall user ratings to Experiments. Please refer to “2. Crossed Ablation of Motion and Reward Revision” in our aggregate comment to all reviewers for detailed settings and results.

W2. Robot diversity

Original weakness: Although the experiments cover 19 tasks, they involve only four robot embodiments. There remains limited support for generalization to substantially different morphologies and mechanical configurations.

Thank you for the question. Although our experiments involve only four robots, they span two intersecting dimensions— existing versus AI-generated platforms, and bipedal versus quadrupedal configurations —forming a complementary and representative evaluation set.

Platform sourceBipedal configurationQuadrupedal configuration
Existing robot platformsMicroDuckUnitree Go2
AI-generated robotsOstrichGorilla

These platforms differ in appearance as well as kinematic structure, support patterns, and whole-body coordination requirements. MicroDuck represents a compact bipedal platform, while Go2 represents a quadrupedal locomotion platform. Ostrich requires coordination of the neck, legs, and tail, whereas Gorilla involves upper-limb motion and support transitions involving the hands and feet. These experiments therefore cover different joint arrangements, contact patterns, and motion coordination requirements, providing evidence for the method’s applicability across the tested morphologies and mechanical configurations.

W2. Does the model comparison hold across repeated generations?

Original weakness: Also, section 3.2 compares only one gorilla-inspired design from each of four models.

We conducted three independent generations per model and expanded the comparison to six models across the GPT, Gemini, and DeepSeek families (Table R5). GPT-6 Astra retains the highest mean number of passed checks, providing further support for the original comparison beyond a single sample per model. We have added these results to Appendix I. Please refer to “3. Repeated Robot Generation Across Model Families” in our aggregate comment to all reviewers for details.

W3. What human intervention was required before fabrication?

Original weakness: Insufficient detail on human intervention.

Please refer to “4. Human Inspection and Corrections Before Fabrication” in our aggregate comment to all reviewers.

Thanks for your response. The response doesn't match the question, so I will keep the score.

Thank you for your feedback. We sincerely apologize for any misunderstanding or misalignment in our previous response. Could you kindly clarify which specific aspects of your question we failed to address? We would greatly appreciate the opportunity to better understand your concerns and provide a more relevant clarification.

Reviewer_DdRaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper looks at "agentic robot creation" in two parts. (i) Embodiment creation: a multimodal model (GPT-6 Astra) receives a reference image, task descriptions, actuator CAD/specifications and assembly examples, and generates a robot as STEP/STL/URDF. The authors propose 23 pre-fabrication checks in three groups: design and assembly (10), mechanical and actuation feasibility (6), and motion and task execution (7). They use these checks to compare one gorilla design from each of four GPT variants (Section 3, Table 1, Appendix I). (ii) Behavior creation: RoboForge, an agentic loop. Given a fixed embodiment and task, the agent writes a procedural reference motion (root trajectory, end-effector targets, selected joint trajectories, IK for the remaining joints), configures a DeepMimic-style tracking reward, trains a PPO policy, inspects the rollouts, and revises the motion, the reward, or both, with up to three trials (Section 4). The method is evaluated on 19 tasks across four robots (Go2, MicroDuck, and generated Ostrich and Gorilla). Baselines are an Eureka-style reward-only refinement, a "One-shot Motion" baseline (RoboForge's first trial), and an unguided GPT agent. Evaluation uses a 57-participant user study and four reference-independent metrics: yaw error, foot slip, action second difference and joint-acceleration RMS (Tables 2–4, Appendices E–H). The fabricated Gorilla runs walking and waving policies on hardware, shown qualitatively (Section 6, Figure 6).

The supported findings are as follows. RoboForge gets the highest cross-robot mean user rating (3.97 vs. 3.60 for One-shot, 3.61 for unguided GPT and 2.10 for Eureka-style) and the best aggregate physical metrics. The agent revises the reference motion in 11 of 19 tasks. The aggregates I recomputed from Tables 9–12 match the main-text numbers. Several broader claims go beyond this evidence: that the metrics "characterize the mechanical knowledge" of MLLMs, that motion-in-the-loop refinement is the cause of the gains, and that behaviors transfer to the real robot. The weaknesses below explain why.

Critical strengths
  1. The paper covers the full pipeline end to end: image and text intent, generated CAD/URDF, simulated policy learning, and a 3D-printed robot that runs learned walking and waving behaviors (Section 6, Appendices A.3–A.4, Figures 6 and 8). Few papers connect generative embodiment design to deployed behavior, so this is a useful system demonstration for the robot-creation community.
  2. The behavior evaluation is designed with care. It avoids reward and tracking-error metrics that cannot be compared across methods. Instead it uses reference-independent physical metrics, defined precisely in Appendix F.2, and a paired user study with randomized, anonymized videos and a stated rubric (Appendix E). The aggregation rules are fully specified (Eqs. 6–10, 15). I recomputed the main-table aggregates (Table 2 from Table 9; Tables 3 and 4 from Tables 10 and 12) and they are internally consistent. Training budgets for GPT, RoboForge and Eureka-style are explicitly matched at 9,000 PPO iterations per task (Eq. 5).
  3. The pre-fabrication checklist is detailed and concrete, with tolerance stacks, clearance budgets, load-path and fastener-chain checks and explicit thresholds (Appendix I.2–I.4). Each check has a per-design pass/fail verdict (Table 13). This is a practical and transparent first step towards screening generated hardware before printing.
Critical weaknesses
  1. The central causal claim is not isolated by ablation. The key claim is that keeping motion in the loop, i.e. revising the reference motion and not only the reward, drives the improvements (Sections 1, 4.2, 5.3). No experiment compares RoboForge against the same pipeline with reward-only revision (fixed motion) or motion-only revision (fixed reward). Section 5.3 reports only how often revisions occur (15 motion revisions in 11/19 tasks) and one qualitative example (Figure 4). The paper also does not report how many tasks ended after the first trial, which would make RoboForge identical to One-shot (Appendix D.2). The two comparisons that are reported do not answer the question:

    • Eureka-style differs from RoboForge in having no reference motion at all. Its gap (2.10 vs. 3.97) mostly confirms the known benefit of imitation-based guidance for stylized tasks.
    • The unguided GPT baseline (Section 5.4, Appendix D.1) is not given the DeepMimic-style tracking scaffold. Its gap therefore mixes the refinement procedure with the tooling.
  2. The One-shot comparison has unequal compute and no variance. One-shot is RoboForge's first trial and receives at most 3,000 PPO iterations, while RoboForge can use up to 9,000 (Appendix D.4). RoboForge's later trials retrain from scratch, so part of the 3.60 to 3.97 gain could come from extra restarts and seed diversity rather than diagnosis-driven revision. A matched baseline is needed, for example One-shot with three independent trials or retraining plus agent selection. Variance is also missing:

    • Each method–task pair is evaluated from one training run (default seed 0, Appendix B.4) and one evaluation rollout (Appendix F.2).
    • No statistical tests are reported for the user ratings, although the per-cell standard deviations are around 0.6–1.4 (Table 9) and ratings are clustered within raters.

    The per-robot results are mixed. RoboForge is lower than One-shot on Ostrich (Table 2) and loses to One-shot on Gorilla action variation and joint acceleration (Table 10). Against unguided GPT, it has worse action variation on 3 of 4 robots and worse joint acceleration on 2 of 4 (Table 12). The aggregate advantages in Table 4 for these two metrics come from a single outlier, Go2 (GPT 0.0698 rad² and 68.9 rad/s²). The statement that the structured pipeline is better "on all four" metrics therefore overstates the evidence.

  3. The embodiment study is too thin for its claims, and the checks are not validated. Several claims, such as that the paper "systematically compare[s] different MLLMs" and "characterize[s] the mechanical knowledge they already possess" (Section 1), rest on very limited evidence:

    • Each model is represented by one design, all models are from one vendor, and only one morphology (gorilla) and two task motions are used (Section 3.2, Appendix I.1).
    • The check programs are run by GPT-6 Astra, which is also the generator of the best-scoring design (Appendix I.1). This raises a self-evaluation concern that the paper does not discuss.
    • The checks' predictive value is never validated against physical outcomes.
    • Even the best design fails 8 of 23 checks, including fit tolerance, assembly/tool access, fastener interfaces, static assembly clearance, simulation tracking, load-path strength and cable routing (Table 13). It was "manually correct[ed]" before fabrication (Section 3.2), but the nature and extent of the corrections are not described. This weakens the claim that designs passing the checks can be fabricated (Section 1). It also leaves unclear how much of the fabricated Gorilla is agent-generated.
    • The verdicts look internally inconsistent: for GPT-6 Astra, "Fastener interfaces" (screw length, engagement, bottoming) fails, while "Fastener length chain" (span, engagement, bottoming) passes.
  4. Real-world validation is qualitative only, and sim-to-real details are missing. Section 6 shows image sequences (Figure 6) of two of the four Gorilla tasks, with no success rates, number of trials or sim/real comparison. Appendix B.2 states that the motion-tracking configurations use no observation noise, no reset perturbations, fixed contact parameters and no external pushes. The paper does not explain how direct deployment succeeds without domain randomization or actuator modeling, so the transfer claim in the abstract cannot be assessed. Reproducibility also depends on a proprietary model and on agent prompts and acceptance criteria that are not given in the paper.

Minor issues:

  • Appendix G.1 states that Go2's gain is driven by walking "whose mean changes from 4.56 to 4.33". Those are the non-walking values; walking goes from 3.00 to 3.76 (Table 9).
  • Check IDs in Table 13 are inconsistent (DA01–DA06 alongside A05–A09, B05, C02, C05).
  • The category order differs between Section 3.1/Table 1 and Appendix I.1.
  • Section 5.4 is titled "Can GPT discover the refinement strategy", while Section 5 phrases it as "recover".
Questions for authors
  1. Can you report a 2×2 ablation on the same 19 tasks and budget: reward-only revision vs. motion-only vs. both vs. neither? The paper's main mechanistic claim depends on this.
  2. Can you add a compute-matched One-shot baseline, for example three independent trials with the same reference and reward, selected by the same agent or by the last trial, and give results over several seeds with confidence intervals or paired significance tests for the user ratings? Please also report how many of the 19 tasks stopped after trial 1. If most of the One-shot gap disappears under matched compute, my assessment of the refinement contribution would change substantially.
  3. What exactly was manually corrected in the GPT-6 Astra Gorilla before fabrication? Have any check outcomes been validated against physical outcomes, e.g. failed clearance or fastener checks matching real assembly problems? How do you address the fact that GPT-6 Astra both generated the winning design and executed the checks for all four designs?
  4. How were the hardware policies deployed, and with what success? Please give the number of trials and success rate, any domain randomization or actuator modeling used for the deployed policies (beyond the defaults in Appendix B.2), and whether the handstand and stretch policies were tried on hardware.
2 comments

We thank the reviewer for recognizing our complete pipeline and its value to the robot-creation community. We have corrected the Go2 walking-score comparison in Appendix G, standardized the check identifiers in Appendix I and aligned its category ordering with Embodiment Creation, and unified the “discover” wording in Experiments. We address the main concerns below.

W1 + Q1. What does the 2×2\boldsymbol{2 \times 2} ablation show about the contribution of motion revision?

Original question: Can you report a 2×2\boldsymbol{2 \times 2} ablation on the same 19 tasks and budget: reward-only revision vs. motion-only vs. both vs. neither?

Thank you for the question. We added Best-of-3 One-shot, Only Reward, and Only Motion to form a 2×2\boldsymbol{2 \times 2} ablation with Ours, using matched maximum training budgets and three RL seeds across all 10 tasks on Go2 and Ostrich. Physical metrics and user ratings are reported in Tables R3 and R4, respectively. Ours achieves better means on all four physical metrics than Best-of-3 One-shot and Only Reward, improves three of the four metrics over Only Motion, and receives the highest mean user rating among the four settings.

These results support the additional value of allowing motion revision alongside reward revision. We have added the ablation settings, physical metrics, and overall user ratings to Experiments. Please refer to “2. Crossed Ablation of Motion and Reward Revision” in our aggregate comment to all reviewers (Tables R3–R4).

W2 + Q2. Do the gains persist with matched training budgets and multiple RL seeds?

Original question: Can you add a compute-matched One-shot baseline, for example three independent trials with the same reference and reward, selected by the same agent or by the last trial, and give results over several seeds with confidence intervals or paired significance tests for the user ratings?

Thank you for highlighting the training-budget and seed-variance concerns. We added a Best-of-3 One-shot comparison with matched maximum training budgets across three RL seeds on 10 tasks on Go2 and Ostrich (Tables R3–R4). Even when Best-of-3 One-shot independently generates and trains three candidates and selects the best policy, Ours achieves lower mean values on all four physical metrics and a higher mean user rating. We have added the ablation settings, physical metrics, and overall user ratings to Experiments. See “2. Crossed Ablation of Motion and Reward Revision” in our aggregate comment to all reviewers.

We also added three-seed comparisons of Ours, Eureka-style, and Unguided GPT across the same 10 tasks, reporting means and sample standard deviations across seeds for the physical metrics, together with user ratings (Tables R1–R2). Ours achieves better means on all four physical metrics and the highest mean user rating. We have added these multi-seed comparisons to Appendix G. Please refer to “1. Baseline Comparisons Across RL Seeds” in our aggregate comment to all reviewers.

W2. The Go2 outlier in physical metrics

Original weakness: The aggregate advantages in Table 4 for these two metrics come from a single outlier, Go2 (GPT 0.0698 rad2\boldsymbol{\mathrm{rad}^{2}} and 68.9 rad/s2\boldsymbol{\mathrm{rad}/\mathrm{s}^{2}}).

Table 4 reports overall performance across embodiments under a common evaluation protocol. We apply identical metric definitions and aggregation rules to all methods: action variation and joint acceleration are normalized by joint count, task-level metrics are averaged within each embodiment, and embodiments receive equal weight. All applicable executions are retained, including poorer executions. Consequently, joint count and task duration do not increase an embodiment’s weight through simple accumulation, and both stronger and weaker performance enter the evaluation under the same rules. “Better on all four metrics” refers to this aggregate comparison.

To further assess performance across independent training runs, we conducted three-seed comparisons across 10 tasks on Go2 and Ostrich. Ours achieves lower aggregate means on all four physical metrics and a higher mean user rating than Unguided GPT (Tables R1–R2). These multi-seed results provide further support for the performance advantages of our method. We have added these comparisons to Appendix G.

W3. Does the comparison include multiple designs and model families?

Original weakness: Each model is represented by one design, all models are from one vendor.

Thank you for the suggestion. We expanded the comparison to six models across the GPT, Gemini, and DeepSeek families, with three independent generations per model, reporting the mean and sample standard deviation of the number of passed checks (Table R5). GPT-6 Astra retains the highest mean number of passed checks in the repeated experiments. We have added these results to Appendix I. Please refer to “3. Repeated Robot Generation Across Model Families” in our aggregate comment to all reviewers.

W3 + Q3. Model bias in engineering checks for generated robots

Original question: How do you address the fact that GPT-6 Astra both generated the winning design and executed the checks for all four designs?

Thank you for raising this concern. Verification is indeed performed by an agent; however, the agent writes and executes scripts according to explicit geometric and physical evaluation criteria, obtaining Pass/Fail verdicts through deterministic computations. To examine possible bias arising from model-specific implementation preferences, we asked Gemini 3.8 Flash High to independently reimplement and execute all 23 checks under the same evaluation criteria. Across the same 19 frozen designs, the two implementations agree on 428 of 437 verdicts (97.94%; Table R6). These results suggest that the check outcomes do not exhibit substantial model-specific bias on the evaluated designs. We have added the independent-verification results to Appendix I. Please refer to “5. Model Bias in Engineering Checks for Generated Robots” in our aggregate comment to all reviewers for the experimental details and a concrete disagreement analysis.

W3 + Q3. What was manually corrected?

Original question: What exactly was manually corrected in the GPT-6 Astra Gorilla before fabrication? Have any check outcomes been validated against physical outcomes, e.g. failed clearance or fastener checks matching real assembly problems?

Please refer to “4. Human Inspection and Corrections Before Fabrication” in our aggregate comment to all reviewers.

W3. Why do the two fastener checks yield different verdicts?

Original weakness: The verdicts look internally inconsistent: for GPT-6 Astra, "Fastener interfaces" (screw length, engagement, bottoming) fails, while "Fastener length chain" (span, engagement, bottoming) passes.

Thank you for the question. The two checks have different scopes, which the original manuscript did not explain sufficiently. "Fastener length chain" checks the clamped span, effective thread engagement, and non-bottoming conditions of the servo screw connections. For Astra, direct geometric inspection of the corresponding STEP parts gives the following results; all three connections pass:

Mounting interfaceEffective thread engagementAcceptance criterionResult
Servo horn2.50 mm> 0 and ≤\leq 3.00 mmPass
Motor front case2.50 mm> 0 and < 15.0 mmPass
Motor rear case1.50 mm> 0 and < 15.0 mmPass

"Fastener interfaces" is a broader conjunctive check: it covers geometric conditions across all fastener groups and requires complete procurement information in the BOM. For GPT-6 Astra, the contact-pad adhesive lacks a complete model designation. It therefore fails.

We have clarified the definitions, scopes, and engagement criteria of both checks in Appendix I.

W4 + Q4. Hardware deployment

Original question: How were the hardware policies deployed, and with what success? Please give the number of trials and success rate, any domain randomization or actuator modeling used for the deployed policies (beyond the defaults in Appendix B.2), and whether the handstand and stretch policies were tried on hardware.

We acknowledge the limited coverage of our current hardware validation. Across repeated hardware tests, Gorilla’s walking and waving achieved completion rates close to 100%, with both policies deployed without domain randomization. More demanding skills, such as handstands, involve more complex balance control and contact transitions and are more sensitive to model errors, noise, and contact conditions. Their transfer to hardware therefore faces a larger sim-to-real gap.

This work represents a first step toward connecting robot generation, fabrication, policy learning, and hardware deployment through a GPT-guided pipeline. As future work, we will focus on transferring more demanding skills to hardware, narrowing the sim-to-real gap, and extending quantitative hardware evaluation. We have added these future directions to Conclusion.

Reviewer_Sr3QOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether an AI agent can go from a user's high-level description to a working robot, and splits the problem into two parts.

For embodiment creation, GPT models generate a robot (STEP, STL and URDF files) from a reference image, task descriptions, actuator CAD files and assembly examples. The authors propose pre-fabrication checks in three groups: design and assembly, mechanical and actuation feasibility, and motion and task execution. They use these checks to compare one gorilla design from each of four GPT variants.

For behavior creation, the paper proposes RoboForge. Given a fixed robot and a task, the agent:

  • writes a procedural reference motion (root trajectory, end-effector targets and selected joint trajectories, with IK for the remaining joints);
  • sets up a DeepMimic-style tracking reward and trains a PPO policy;
  • inspects the rollout and decides whether to revise the motion, the reward, or both.

RoboForge uses at most three training trials per task.

On 19 tasks across four robots (Go2, MicroDuck, and generated Ostrich and Gorilla), RoboForge is compared with an Eureka-style reward-refinement baseline and a one-shot motion baseline. The evaluation uses a user study (n = 57), four reference-independent metrics, and a comparison against an unguided GPT agent. Finally, the generated Gorilla is 3D-printed after manual fixes, and walking and hand-waving policies are deployed on it.

Critical strengths

The scope is the main strength. Few papers go all the way from a generated design to a printed robot running learned policies, and even with manual fixes in the loop, the hardware demonstration makes the problem concrete.

The paper also makes a clear practical case that the reference motion should stay revisable once execution feedback is available. Fig. 4 is a good example: the original trot satisfied the task description but was hard for the policy to track, and switching to a four-beat gait fixed this without changing what was asked for.

The pre-fabrication checklist is practical, and Fig. 1 shows it catches failures, such as self-intersections and missing mounting features, that are easy to miss by looking at a render.

Critical weaknesses

1. The new element over prior work is small, and the experiments don't isolate it.

Novelty. The behavior pipeline combines two established lines of work:

  • LLM-generated reference motions for example-guided RL (MimicAgent, which already reports human evaluations favoring this approach over Eureka);
  • Eureka-style refinement from training feedback.

The new element is letting rollout feedback revise the reference motion as well as the reward. This idea also has precedents:

  • language-guided iterative motion refinement for imitation-based control (Kumar et al., 2023, "Words into Action");
  • LLM-specified foot contact patterns as the interface to a locomotion controller (SayTap; Tang et al., 2023), which is essentially what the trot-to-four-beat revision in Fig. 4 changes.

Neither is discussed in the related work. The paper's own Table 4 also suggests the strategy is not hard to find: an unguided GPT agent introduces a reference motion in 16 of 19 tasks and revises it in 7 on its own. What remains specific to RoboForge is the structured diagnosis procedure, so the experiments need to show that this procedure is what drives the gains. In my view they do not, for the reasons below.

Budget and selection. RoboForge trains up to three policies and returns the first one its own reviewer accepts, or the last one if none is accepted. The reviewer checks task compliance, motion quality, support and balance, which is close to what the user study rates. One-shot Motion is the first trial, with no review at all. The overall gain (3.60 to 3.97) therefore mixes more attempts, selection against evaluation-like criteria, and the diagnosis itself. A best-of-three One-shot baseline with the same reviewer would separate these.

Motion versus reward. No baseline keeps the reference motion fixed and refines only the reward. The reference-independent metrics in Table 3 can also be targeted directly by reward terms the agent is free to add during refinement, such as heading tracking, foot-slip penalties and action-rate penalties. One-shot, which uses a reference motion, has worse yaw error than Eureka-style, which does not (0.058 vs. 0.035), and RoboForge's large gains on yaw and foot slip are what one would expect from such reward terms. Since 11 of the 19 tasks had motion revisions, splitting the existing results by what was revised would test the motion-in-the-loop claim directly.

The unguided agent and the diagnosis. The unguided agent already matches One-shot on user rating (3.61 vs. 3.60) and beats it on three of the four metrics. The margin attributable to the structured procedure is therefore 3.61 to 3.97, and the unguided agent's training budget is not stated. Section 4.3 says Section 5.4 evaluates the diagnosis procedure, but 5.4 compares final outcomes only; nothing checks whether the agent's attribution of a failure to the motion or to the reward is correct.

Statistics. Each method contributes one trained policy per task, so the effective sample is 19 paired tasks regardless of the 57 raters, and variance across RL seeds is not captured. No per-task results or significance tests are given. The gains are concentrated on MicroDuck (+0.73) and Gorilla (+0.63), compared with +0.17 on Go2 and −0.05 on Ostrich.

2. The embodiment-creation part is thin and not connected to the rest.

Prior work. Text2Robot (Ringel et al., 2025) already goes from a text description to a 3D-printed walking robot, accounting for manufacturability and co-optimizing morphology and gait with evolutionary search and RL. The paper cites it only under morphology generation, but its end-to-end "intent to physical robot" framing needs to be positioned against this work. The pre-fabrication checks themselves read largely as a collection of standard design-for-manufacturing and engineering checks.

Model comparison. Table 1 rests on one gorilla design per model, so it is hard to tell how much of the ranking reflects the models and how much reflects sampling variance in a single generation. All four models are GPT variants. I recognize that only a handful of frontier models are currently capable of this kind of CAD generation, and access to them may be limited. If the authors do have access to a comparably capable model from another family, including it would make the comparison much more informative. If not, the conclusions are best read as being about these GPT variants rather than about current MLLMs in general, and the text could be scoped accordingly. The generation pipeline, including the prompts and assembly examples, was presumably developed with GPT-6 Astra, which may also favor it.

Buildability. Even the best design fails 8 of the 23 checks (4/10, 2/6 and 2/7 across the three categories). The fabricated robot is that design after manual corrections, and the extent of those corrections is not described. The paper therefore never shows a generated design passing the checks unaided. Nor does it show that the checks predict buildability: no failing design was fabricated, and the checks were not compared against expert judgment. The introduction's statement that "designs passing these checks can be assembled and operated" is not what the experiments show.

Missing loop. The generation, verification and revision loop in the title exists only on the behavior side. On the embodiment side, verification is followed by human fixes, and since the robot is held fixed in RoboForge, behavior failures never feed back into the design.

Hardware. The hardware demonstration covers the two simplest behaviors (walking and waving) on a tethered robot, with qualitative frames only. The handstand showcased in simulation is not attempted on hardware.

Questions for authors
  1. How do a best-of-three One-shot baseline (same reviewer, same training budget) and a fixed-motion, reward-only refinement baseline compare to RoboForge? These are the results most likely to change my assessment.
  2. Could you report per-task results split by what was revised (motion, reward, both, or nothing), with spread across RL seeds? For tasks with no revision, is the final policy identical to One-shot's? What training budgets did Eureka-style and the unguided GPT agent receive, and were raters blind to the method?
  3. What exactly was manually corrected on the GPT-6 Astra design before fabrication, and did the corrected design pass all 23 checks? Are the checks computed by scripts or judged by the agent?
  4. Did you generate multiple designs per model? Even a few samples per model would show how much of Table 1 reflects the model rather than sampling variance. If a comparably capable model from another family is available to you, adding it would help; otherwise, scoping the claims to GPT variants is fine. Was the generation pipeline tuned using GPT-6 Astra?
3 comments

We thank the reviewer for recognizing our work’s scope and the value of feedback-guided motion revision. We clarify the contributions of motion revision, reward revision, and candidate search, and the evidence from robot generation, manual corrections, and hardware validation.

W1. What distinguishes RoboForge from prior work?

Original weakness: Neither is discussed in the related work (Words into Action & SayTap)

We thank the reviewer for highlighting these two relevant works. We have added these works to Related Work and clarified their connections to and differences from our approach. Words into Action involves humans updating semantic instructions based on policy execution results and allows the original task’s semantic goal to be modified during interaction. SayTap uses foot contact patterns specified through language as an interface for quadrupedal locomotion control.

In our pipeline, an agent analyzes rollouts of trained policies and proposes revisions, jointly adjusting the reference motion and reward design while treating the initial task semantics as a fixed constraint. Our central focus is how execution feedback can guide revisions to the reference motion and reward design to improve policy execution quality on the original task while preserving its intended meaning.

W1 + Q1. How does RoboForge compare with Best-of-3 One-shot and fixed-motion baselines?

Original question: How do a best-of-three One-shot baseline (same reviewer, same training budget) and a fixed-motion, reward-only refinement baseline compare to RoboForge?

We thank the reviewer for this question. We added Best-of-3 One-shot, Only Reward, and Only Motion with matched maximum training budgets and conducted comparisons across three RL seeds on all 10 tasks on Go2 and Ostrich (Tables R3–R4). Ours achieves lower mean values on all four physical metrics than Best-of-3 One-shot and Only Reward, outperforms Only Motion on three of the four metrics, and attains the highest mean user rating among the four settings.

These results support the benefit of refinement guided by execution feedback over independent candidate search, as well as the additional value of allowing motion revision alongside reward revision. We have added the ablation settings, physical metrics, and overall user ratings to Experiments. For detailed experimental settings and means and standard deviations across seeds, please refer to “2. Crossed Ablation of Motion and Reward Revision” in our aggregate comment to all reviewers (Tables R3–R4).

W1 + Q2. How variable are the results across RL seeds?

Original weakness: Variance across RL seeds is not captured.

We added experiments with three RL seeds for Ours, Eureka-style, and Unguided GPT across all 10 tasks on Go2 and Ostrich, reporting the means and sample standard deviations of the physical metrics. Ours achieves lower mean values on all four physical metrics than both baselines (Table R1) and the highest mean user rating in the corresponding user study (Table R2). We have added these multi-seed comparisons to Appendix G.

These results supplement the original performance comparison with evidence across independent training runs, while the standard deviations quantify variability in training outcomes. Please refer to “1. Baseline Comparisons Across RL Seeds” in our aggregate comment to all reviewers for detailed settings and results.

W1. What does the case study show about the benefits of structured diagnosis?

Original weakness: Section 4.3 says Section 5.4 evaluates the diagnosis procedure, but 5.4 compares final outcomes only; nothing checks whether the agent's attribution of a failure to the motion or to the reward is correct.

Thank you for raising this point. Experiments presents a comparison of diagnosis-guided motion revisions and refers to Appendix H, which shows the corresponding final policy executions. In the Go2 low-walk case, both approaches identify foot sliding and increase swing-foot clearance. Our structured procedure further considers whole-body coordination, adjusting body motion and the supporting limbs to coordinate swing with posture maintenance and support transfer.

Appendix H shows the downstream consequence: our learned policy more clearly realizes distinct stepping and coordinated support transfer, whereas increasing swing-foot clearance alone does not fully resolve the whole-body gait. Together, these results provide case-level evidence for the effectiveness of the structured diagnosis-and-revision procedure. We have revised the method description and Experiments to make this connection explicit, supported by the policy-execution analysis in Appendix H.

W1 + Q2. Training budgets and blinded evaluation

Original question: What training budgets did Eureka-style and the unguided GPT agent receive, and were raters blind to the method?

Thank you for the question. As specified in Appendix D, Eureka-style and Unguided GPT each receive a maximum formal training budget of 9,000 PPO iterations per task, matching Ours.

Raters were blind to method identity. As described in Appendix E, the four policy videos for each task were presented in randomized order and identified only by anonymous labels A–D.

W2. How do the metrics differ from standard engineering checks?

Original weakness: The pre-fabrication checks themselves read largely as a collection of standard design-for-manufacturing and engineering checks.

We agree that the 23 metrics build on standard design-for-manufacturing and engineering checks. Our contribution is to organize these checks, with input from mechanical engineering experts, into a task-conditioned pre-fabrication evaluation framework for robot-building agents. Under matched reference morphology, task requirements, and actuator assets, it jointly assesses design and assembly, mechanical and actuation feasibility, and motion and task execution, extending evaluation to whether generated embodiments can support user-specified behaviors.

W2 + Q4. Repeated generation, model diversity, and pipeline tuning

Original question: Did you generate multiple designs per model? Was the generation pipeline tuned using GPT-6 Astra?

Thank you for the suggestion. We expanded the comparison to six models across the GPT, Gemini, and DeepSeek families, with three independent generations per model, reporting the mean and sample standard deviation of the number of passed checks (Table R5). GPT-6 Astra retains the highest mean number of passed checks in the repeated experiments. We have added these results to Appendix I. Please refer to “3. Repeated Robot Generation Across Model Families” in our aggregate comment to all reviewers.

The prompts and engineering guidelines for the generation pipeline were manually authored by mechanical engineering experts and were not specifically tuned for GPT-6 Astra. All models used the same prompt templates, reference images, actuator assets, task requirements, and assembly examples.

W2 + Q3. What was manually corrected, and did the design pass the checks?

Original question: What exactly was manually corrected on the GPT-6 Astra design before fabrication, and did the corrected design pass all 23 checks? Are the checks computed by scripts or judged by the agent?

Please refer to “4. Human Inspection and Corrections Before Fabrication” in our aggregate comment to all reviewers.

Verification is performed by an agent that writes and executes scripts according to explicit evaluation specifications. Pass/Fail outcomes are obtained through deterministic geometric and physical computations. Our independent reimplementation experiment is described in “5. Model Bias in Engineering Checks for Generated Robots” (Table R6).

W2. Hardware validation

Original weakness: The hardware demonstration covers the two simplest behaviors (walking and waving) on a tethered robot, with qualitative frames only. The handstand showcased in simulation is not attempted on hardware.

We provide full videos of Gorilla’s walking and waving policies executing on hardware in the supplementary material. The main paper presents key frames from these executions.

We acknowledge the limited coverage of the current hardware validation. More demanding skills, such as handstands, involve more complex balance control and contact transitions and are more sensitive to model errors, noise, and contact conditions. Their transfer to hardware therefore faces a larger sim-to-real gap. Future work will focus on hardware transfer, validation, and quantitative evaluation of more demanding skills. We have added these future directions to Conclusion.

W2. Design–behavior feedback loop

Original weakness: Behavior failures never feed back into the design.

We acknowledge the limitations of the current design–behavior loop. This work represents a first step toward connecting robot generation, fabrication, policy learning, and hardware deployment. Currently, the embodiment remains fixed during behavior learning, while execution feedback guides motion and reward revisions. As future work, we will explore structural revisions driven by execution failures, advancing bidirectional refinement of structure and control. We have added these future directions to Conclusion.

Comments on the paper

5 comments

We thank all reviewers for their constructive comments and suggestions, which have helped us improve the manuscript. They recognize our scope as “the main strength” (Sr3Q) and our pipeline as a “useful system demonstration for the robot-creation community” (DdRa). They find the feedback loop “well motivated” (zYmR) and highlight the “clear practical case” for keeping reference motions revisable under execution feedback (Sr3Q). They describe our evaluation as “comprehensive and multi-faceted” (zYmR) and our pre-fabrication checklist as “detailed and concrete” (DdRa). Below, we address shared concerns through multi-seed comparisons, budget-matched ablations, repeated multi-model generation, and independent reimplementation of the checks. We also clarify human intervention and hardware validation and respond to individual comments.

1. Baseline Comparisons Across RL Seeds (Sr3Q, DdRa)

We thank the reviewers for raising concerns about variability across random seeds and training-budget comparability. Given the time constraints of the rebuttal period, we added three-RL-seed comparisons of Ours, Eureka-style, and Unguided GPT across all 10 tasks on Go2 and Ostrich. Table R1 reports the means and sample standard deviations of the physical metrics to quantify variability across independent training runs.

To address the lower training budget of the original One-shot baseline, we added Best-of-3 One-shot as a budget-matched baseline. It independently generates and trains three candidate solutions before selecting the final policy. Its maximum training budget matches Ours at 9,000 PPO iterations per task. The corresponding physical metrics and user ratings are reported in Tables R3 and R4, respectively; this section focuses on the multi-seed comparison of Ours, Eureka-style, and Unguided GPT.

Table R1. Physical metrics across all 10 tasks on Go2 and Ostrich, reported as mean ± sample standard deviation across three RL seeds.

BaselineYaw MAE (rad) ↓\downarrowFoot-slip RMS (m/s) ↓\downarrowAction Var. (rad2\mathrm{rad}^{2}) ↓\downarrowJoint Acc. RMS (rad/s2\mathrm{rad}/\mathrm{s}^{2}) ↓\downarrow
Unguided GPT0.0145 ± 0.006430.0539 ± 0.01430.0159 ± 0.016740.1 ± 5.45
Eureka-style0.0304 ± 0.01470.149 ± 0.06740.0532 ± 0.0238124 ± 18.3
Ours0.0126 ± 0.003210.0347 ± 0.02740.000419 ± 0.000073013.8 ± 2.05

Table R2. User study ratings for Ours, Eureka-style, and Unguided GPT (1–5; higher is better). The final column is the equally weighted mean of the two robot-level mean ratings.

BaselineGo2OstrichOverall
Unguided GPT3.57 ± 1.093.25 ± 0.773.41
Eureka-style1.90 ± 0.501.45 ± 0.641.68
Ours3.87 ± 1.093.95 ± 0.993.91

In the results aggregated across both robots, Ours achieves lower means on all four physical metrics than Unguided GPT and Eureka-style. Compared with Unguided GPT, action variation and joint acceleration are reduced by approximately 97% and 66%, respectively. In the user study, Ours achieves the highest mean rating on each robot, with an equally weighted cross-robot mean of 3.91, compared with 3.41 for Unguided GPT and 1.68 for Eureka-style. These results supplement the original performance comparison with evidence across independent training runs, while the standard deviations quantify variability in training outcomes. We have added these multi-seed comparisons to Appendix G.

2. Crossed Ablation of Motion and Reward Revision (Sr3Q, DdRa, zYmR)

We conducted a 2×2\boldsymbol{2 \times 2} crossed ablation along two dimensions: whether motion revision and reward revision are allowed based on execution feedback. Given the time constraints of the rebuttal period, the additional experiments focus on the existing quadrupedal robot Unitree Go2 and the AI-generated bipedal robot Ostrich, covering all 10 tasks and aggregating results across three RL seeds.

The four settings are:

MethodMotion revision from execution feedbackReward revision from execution feedback
Best-of-3 One-shot×\times×\times
Only Reward×\times✓\checkmark
Only Motion✓\checkmark×\times
Ours✓\checkmark✓\checkmark

Best-of-3 One-shot independently generates and trains three candidate solutions, after which an independent agent selects the best policy according to the acceptance criteria. Only Reward freezes the accepted reference motion while allowing revisions to the reward. Only Motion fixes the reward while allowing reference-motion revisions. Ours allows both types of revision. All methods use the same robot-specific training templates and share the same maximum training budget: three training trials per task, each with up to 3,000 PPO iterations, for a total of at most 9,000 iterations.

Table R3: Physical metrics across all 10 tasks on Go2 and Ostrich, reported as mean ± sample standard deviation across three RL seeds.

BaselineYaw MAE (rad) ↓\downarrowFoot-slip RMS (m/s) ↓\downarrowAction Var. (rad2\mathrm{rad}^{2}) ↓\downarrowJoint Acc. RMS (rad/s2\mathrm{rad}/\mathrm{s}^{2}) ↓\downarrow
Best-of-3 One-shot0.0390 ± 0.01480.0605 ± 0.002260.00238 ± 0.00010326.9 ± 2.17
Only Reward0.0307 ± 0.008690.0374 ± 0.001840.00109 ± 0.00081716.2 ± 3.24
Only Motion0.0182 ± 0.006170.0335 ± 0.002090.00145 ± 0.00046016.7 ± 3.74
Ours0.0126 ± 0.003210.0347 ± 0.02740.000419 ± 0.000073013.8 ± 2.05

Table R4: User study ratings (1–5; higher is better). The final column is the equally weighted mean of the two robot-level mean ratings.

BaselineGo2OstrichOverall
Best-of-3 One-shot3.85 ± 0.913.93 ± 0.963.89
Only Reward3.65 ± 1.013.87 ± 0.993.76
Only Motion3.53 ± 1.363.47 ± 0.643.50
Ours3.87 ± 1.093.95 ± 0.993.91

Compared with Only Reward, which fixes the reference motion, Ours achieves lower means on all four physical metrics and higher mean user ratings on both robots. These results support the additional value of allowing motion revision while retaining reward revision. Under matched maximum training budgets, Ours also achieves lower means on all four physical metrics than Best-of-3 One-shot, supporting the benefit of execution-feedback-guided revision over best-of-N candidate search.

In the user study, Ours achieves the highest mean rating among the four settings on each robot, with an equally weighted cross-robot mean of 3.91. We have added the ablation settings, physical metrics, and overall user ratings to Experiments.

3. Repeated Robot Generation Across Model Families (Sr3Q, DdRa, zYmR)

We added repeated generation experiments across model families to assess sampling variability. We conducted three independent generations per model and expanded the comparison to six models across the GPT, Gemini, and DeepSeek families. Because GPT-5.5 is no longer available through its API, GPT-6.1 Sol replaces that baseline in the repeated experiments. All experiments use the same prompt templates, reference images, servo assets, task requirements, and physical simulation parameters. Table R5 reports results.

Table R5. Repeated Gorilla generation results, reported as mean ± sample standard deviation over three independent generations per model.

ModelPassed checks ↑\uparrow
GPT-6 Astra11.33 ± 3.21
GPT-6.1 Sol9.00 ± 1.73
GPT-5.6 Sol5.33 ± 1.53
GPT-5.6 Terra5.00 ± 1.00
DeepSeek 4.1 Flash Max5.00 ± 1.73
Gemini 3.8 Flash High4.67 ± 0.58

The robot generated by GPT-6 Astra and used for fabrication is included among the model’s three independently generated samples and passes 15/23 checks. Across repeated generations, Astra retains the highest mean number of passed checks, providing additional evidence for the original comparison and reducing its reliance on a single sample per model. We have added the repeated-generation results to Appendix I.

4. Human Inspection and Corrections Before Fabrication (Sr3Q, DdRa, zYmR)

We thank the reviewers for requesting clarification about human intervention and the relationship between check outcomes and fabrication. Our original descriptions of the evaluation metrics and the statement that we “manually correct the remaining design issues” introduced ambiguity. The 23 pre-fabrication checks comprise 15 mandatory checks and 8 supplementary checks. The mandatory checks cover geometric validity and printability, consistency of exported assets and mechanical models, mounting interfaces and fastener engagement, local structural margins, actuator limits, and basic task execution. The GPT-6 Astra design selected for fabrication had already passed all 15 mandatory checks before manual intervention.

Our manual correction was limited to completing procurement information in the bill of materials (BOM) as an additional step to simplify procurement and fabrication. After completing the BOM, we reran the verification scripts under the same evaluation criteria. The design then passed 16 of the 23 checks, comprising all 15 mandatory checks and one supplementary check. Specifically, the Fastener interfaces check changed from Fail to Pass: all 40 CAD counterbores already satisfied the geometric requirements, and the original failure arose from missing part numbers in the BOM. Physical assembly confirmed that these interfaces could be assembled with the selected fasteners.

The remaining seven failed checks are supplementary checks concerning conservative engineering margins or finer execution-quality requirements. For example, the cable-routing check adopts conservative bend-radius requirements; during fabrication, we selected thinner flexible cables and successfully routed and connected them. The Simulation tracking check requires maximum joint tracking error of no more than 5∘\boldsymbol{5^\circ} and maximum root-position error of no more than 5 mm. These thresholds assess how precisely the executed motion follows the reference, supplementing the separate checks of whether the robot can complete the requested basic motions.

We have clarified the distinction between mandatory and supplementary checks in Embodiment Creation and Appendix I, and documented BOM completion, the updated check results, and physical assembly observations in Appendix A.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.