1. The new element over prior work is small, and the experiments don't isolate it.
Novelty. The behavior pipeline combines two established lines of work:
- LLM-generated reference motions for example-guided RL (MimicAgent, which already reports human evaluations favoring this approach over Eureka);
- Eureka-style refinement from training feedback.
The new element is letting rollout feedback revise the reference motion as well as the reward. This idea also has precedents:
- language-guided iterative motion refinement for imitation-based control (Kumar et al., 2023, "Words into Action");
- LLM-specified foot contact patterns as the interface to a locomotion controller (SayTap; Tang et al., 2023), which is essentially what the trot-to-four-beat revision in Fig. 4 changes.
Neither is discussed in the related work. The paper's own Table 4 also suggests the strategy is not hard to find: an unguided GPT agent introduces a reference motion in 16 of 19 tasks and revises it in 7 on its own. What remains specific to RoboForge is the structured diagnosis procedure, so the experiments need to show that this procedure is what drives the gains. In my view they do not, for the reasons below.
Budget and selection. RoboForge trains up to three policies and returns the first one its own reviewer accepts, or the last one if none is accepted. The reviewer checks task compliance, motion quality, support and balance, which is close to what the user study rates. One-shot Motion is the first trial, with no review at all. The overall gain (3.60 to 3.97) therefore mixes more attempts, selection against evaluation-like criteria, and the diagnosis itself. A best-of-three One-shot baseline with the same reviewer would separate these.
Motion versus reward. No baseline keeps the reference motion fixed and refines only the reward. The reference-independent metrics in Table 3 can also be targeted directly by reward terms the agent is free to add during refinement, such as heading tracking, foot-slip penalties and action-rate penalties. One-shot, which uses a reference motion, has worse yaw error than Eureka-style, which does not (0.058 vs. 0.035), and RoboForge's large gains on yaw and foot slip are what one would expect from such reward terms. Since 11 of the 19 tasks had motion revisions, splitting the existing results by what was revised would test the motion-in-the-loop claim directly.
The unguided agent and the diagnosis. The unguided agent already matches One-shot on user rating (3.61 vs. 3.60) and beats it on three of the four metrics. The margin attributable to the structured procedure is therefore 3.61 to 3.97, and the unguided agent's training budget is not stated. Section 4.3 says Section 5.4 evaluates the diagnosis procedure, but 5.4 compares final outcomes only; nothing checks whether the agent's attribution of a failure to the motion or to the reward is correct.
Statistics. Each method contributes one trained policy per task, so the effective sample is 19 paired tasks regardless of the 57 raters, and variance across RL seeds is not captured. No per-task results or significance tests are given. The gains are concentrated on MicroDuck (+0.73) and Gorilla (+0.63), compared with +0.17 on Go2 and −0.05 on Ostrich.
2. The embodiment-creation part is thin and not connected to the rest.
Prior work. Text2Robot (Ringel et al., 2025) already goes from a text description to a 3D-printed walking robot, accounting for manufacturability and co-optimizing morphology and gait with evolutionary search and RL. The paper cites it only under morphology generation, but its end-to-end "intent to physical robot" framing needs to be positioned against this work. The pre-fabrication checks themselves read largely as a collection of standard design-for-manufacturing and engineering checks.
Model comparison. Table 1 rests on one gorilla design per model, so it is hard to tell how much of the ranking reflects the models and how much reflects sampling variance in a single generation. All four models are GPT variants. I recognize that only a handful of frontier models are currently capable of this kind of CAD generation, and access to them may be limited. If the authors do have access to a comparably capable model from another family, including it would make the comparison much more informative. If not, the conclusions are best read as being about these GPT variants rather than about current MLLMs in general, and the text could be scoped accordingly. The generation pipeline, including the prompts and assembly examples, was presumably developed with GPT-6 Astra, which may also favor it.
Buildability. Even the best design fails 8 of the 23 checks (4/10, 2/6 and 2/7 across the three categories). The fabricated robot is that design after manual corrections, and the extent of those corrections is not described. The paper therefore never shows a generated design passing the checks unaided. Nor does it show that the checks predict buildability: no failing design was fabricated, and the checks were not compared against expert judgment. The introduction's statement that "designs passing these checks can be assembled and operated" is not what the experiments show.
Missing loop. The generation, verification and revision loop in the title exists only on the behavior side. On the embodiment side, verification is followed by human fixes, and since the robot is held fixed in RoboForge, behavior failures never feed back into the design.
Hardware. The hardware demonstration covers the two simplest behaviors (walking and waving) on a tethered robot, with qualitative frames only. The handstand showcased in simulation is not attempted on hardware.
We thank the reviewer for recognizing the value of our feedback loop and the breadth of our evaluation. We focus on isolating the contribution of motion revision, clarifying morphological coverage, reporting repeated generation across model families, and documenting human intervention before fabrication.
W1. Does motion revision add value beyond reward-only refinement?
Thank you for the suggestion. We added a 2×2 ablation that independently enables or disables motion revision and reward revision (Tables R3–R4). Compared with Only Reward, which fixes the reference motion, Ours achieves better means on all four physical metrics and a higher mean user rating. This supports the additional value of allowing motion revision while retaining reward revision. We have added the ablation settings, physical metrics, and overall user ratings to Experiments. Please refer to “2. Crossed Ablation of Motion and Reward Revision” in our aggregate comment to all reviewers for detailed settings and results.
W2. Robot diversity
Thank you for the question. Although our experiments involve only four robots, they span two intersecting dimensions— existing versus AI-generated platforms, and bipedal versus quadrupedal configurations —forming a complementary and representative evaluation set.
These platforms differ in appearance as well as kinematic structure, support patterns, and whole-body coordination requirements. MicroDuck represents a compact bipedal platform, while Go2 represents a quadrupedal locomotion platform. Ostrich requires coordination of the neck, legs, and tail, whereas Gorilla involves upper-limb motion and support transitions involving the hands and feet. These experiments therefore cover different joint arrangements, contact patterns, and motion coordination requirements, providing evidence for the method’s applicability across the tested morphologies and mechanical configurations.
W2. Does the model comparison hold across repeated generations?
We conducted three independent generations per model and expanded the comparison to six models across the GPT, Gemini, and DeepSeek families (Table R5). GPT-6 Astra retains the highest mean number of passed checks, providing further support for the original comparison beyond a single sample per model. We have added these results to Appendix I. Please refer to “3. Repeated Robot Generation Across Model Families” in our aggregate comment to all reviewers for details.
W3. What human intervention was required before fabrication?
Please refer to “4. Human Inspection and Corrections Before Fabrication” in our aggregate comment to all reviewers.