acceptodds
Under review as a conference paper at ICLR 2027

Differentiable evolution of agile agents

Abstract

It is usually much easier to compute meaningful policy gradients than meaningful design gradients, especially in jointed rigid bodies that move quickly and gracefully on land. Whereas soft bodies without bones and joints naturally lend themselves to continuous modeling, they are generally slower, weaker, and less interesting in terrestrial environments. Here we demonstrate end-to-end differentiable evolution of articulated agents capable of agile legged locomotion. Starting from a primordial capsule, jointed links emerge and grow and move along the body as others shrink and disappear, across evolutionary time, depending on derivatives of fitness with respect to design variables. These variables were sampled from a distribution to produce a population of designs, which were controlled by a shared universal policy. Gradients were averaged across individuals in the population and across truncated learning windows inside each forward simulation to update the design distribution and universal controller concurrently. This enables the sample-efficient discovery and gradient-based optimization of novel robots with exciting athletic behaviors.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot morphology optimization, morphology-control co-design, differentiable simulation, differentiable physics, reinforcement learning, legged locomotion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_w5vvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FORM, a framework for differentiable morphology–control co-design of articulated rigid-body robots. It combines a continuous relaxation over a fixed kinematic supergraph with short-horizon differentiable simulation, and outperforms CMA-ES on a flat-ground locomotion task.

Critical strengths
  1. This work is well motivated. It provides a useful end-to-end differentiable pipeline. It connects morphology optimization and neural control through differentiable rigid-body simulation, enabling gradient-based optimization of both morphology and control.
  2. The physical modeling is well designed, including contact and friction dynamics, joint limits, and morphology-dependent actuation, with additional treatments to improve numerical stability.
  3. The experiments include comprehensive ablation studies and informative analyses.
Critical weaknesses
  1. The proposed method has limited structural flexibility. Its reliance on a predefined kinematic supergraph restricts the exploration of more complex topologies.
  2. Comparisons with existing gradient-based co-design methods are insufficient. Methods such as Transform2Act and BodyGen also optimize structural design through policy gradients, but are not included in the evaluation. Such comparisons would help establish FORM's advantages over existing approaches.
  3. The experimental evaluation is limited to a single flat-ground locomotion task. It remains unclear whether the observed improvements generalize beyond thissetting and how the method performs on more complex tasks.
Questions for authors
  1. FORM jointly optimizes limb presence, geometry, and attachment, while the presence parameter also affects actuation and observations. Could the authors provide more insight into how these different factors contribute to the evolved morphologies and performance? In particular, are the improvements mainly driven by changes in effective topology or by continuous adjustments to geometry and actuation?
  2. FORM adaptively increases the center fraction and ends with an all-center phase, whereas CMA-ES uses a fixed fraction of 0.125. With the same fixed fraction, FORM reaches only 1.37 (Fig. 5a), below CMA-ES’s 2.11 (Fig. 3). . Could the authors discuss how much of FORM’s advantage comes from this schedule rather than from the pathwise gradients?
7 comments
AuthorsOpenReview ↗

Response to Reviewer w5vv (Part 1/2)

We thank the reviewer for their thoughtful questions, constructive suggestions, and positive assessment of our motivation and contributions. We have conducted several additional experiments to address the reviewer’s comments and have revised the manuscript accordingly.


W1: The proposed method has limited structural flexibility. Its reliance on a predefined kinematic supergraph restricts the exploration of more complex topologies.

Although FORM performs optimization within a predefined candidate supergraph, it is not restricted to the six-appendage configuration used in our main experiments.

To demonstrate this flexibility, we have added Appx. K: Alternative Candidate Supergraphs (Figs. 15–17), where we extend FORM to different supergraph sizes and more complex branching structures. As described in the revised appendix:

Beyond the original design that can express up to 6 appendages, we test supergraphs that can express up to 4 or 8 appendages. We also consider a hierarchical supergraph in which each of the six original proximal links can support an additional two-link branch.

We acknowledge that FORM does not dynamically add new nodes or edges during training. To clarify this distinction, we added the following discussion to Appx. K:

FORM instead performs differentiable co-design within a configurable supergraph, allowing different effective morphologies to emerge without discrete graph-editing operations.


W2: Comparisons with existing gradient-based co-design methods are insufficient. Methods such as Transform2Act and BodyGen also optimize structural design through policy gradients, but are not included in the evaluation. Such comparisons would help establish FORM's advantages over existing approaches.

Understood. We have added comparisons with BodyGen and Transform2Act across eight matched initialization seeds (Sects. 3.1–3.2, Figs. 3 and 5).

As described in the revised Sect. 3.1:

We adapt BodyGen and Transform2Act to the same candidate supergraph and simulation environment, retaining their respective policy architectures, PPO objectives, and credit-assignment mechanisms while omitting discrete topology construction.

We acknowledge that these adaptations omit the original methods' discrete topology-editing ability and therefore compare optimization within a shared candidate supergraph.

The revised Sect. 3.2 reports:

At the matched wall-clock budget of 8.46 hours, FORM reaches a mean best-so-far return of 5.77 ± 0.85, compared with 2.32 ± 0.20 for CMA-ES, 2.13 ± 0.35 for BodyGen, and 2.29 ± 0.31 for Transform2Act (mean ± s.e. across eight seeds).

FORM achieves higher returns in all eight matched runs against each baseline (Figure 3C; (p=0.0078) for each paired comparison). Figure 5 further compares the selected morphologies.

We also explicitly acknowledge that:

Despite its higher mean performance, FORM exhibits greater variability across seeds than the baselines.


W3: The experimental evaluation is limited to a single flat-ground locomotion task. It remains unclear whether the observed improvements generalize beyond this setting and how the method performs on more complex tasks.

To explore FORM beyond flat-ground locomotion, we have added a new Appx. L: “Preliminary stair-climbing experiment” (Fig. 19). We evaluate FORM in an environment with 16 ascending steps of 4 cm each, incorporating terrain-aware observations while retaining the original optimization procedure and training budget.

For convenience, we quote the key findings from the revised manuscript:

Across eight matched seeds, FORM achieves a higher mean best-so-far return than CMA-ES at the common wall-clock budget of 10.72h (2.54 vs. 1.89 m; Fig. 19B). This advantage is also observed under the matched agent-step budget (Fig. 19A). These preliminary results suggest that FORM can effectively optimize morphology and control in more challenging locomotion environments beyond flat ground.

We view this preliminary stair-climbing experiment as an encouraging first step toward more complex, contact-rich tasks. Such environments pose challenges for gradient-based co-design, as the benefits of morphology changes may depend on future contact interactions that are difficult to capture with short-horizon gradients [1] . Building on these initial results, we aim to improve gradient guidance and extend FORM to more challenging tasks, including manipulation, in future work.

References:

[1] Luo, Jing Yuan, et al. (2025) ICRA. https://ieeexplore.ieee.org/document/11127448

AuthorsOpenReview ↗

Response to Reviewer w5vv (Part 2/2)

Q1: FORM jointly optimizes limb presence, geometry, and attachment, while the presence parameter also affects actuation and observations. Could the authors provide more insight into how these different factors contribute to the evolved morphologies and performance? In particular, are the improvements mainly driven by changes in effective topology or by continuous adjustments to geometry and actuation?

In FORM, effective topology and geometry cannot be fully disentangled: geometric changes can alter ground contact even with fixed presence, while presence also affects limb geometry, actuation, and observations. Nevertheless, presence is most directly associated with limb deployment, whereas the remaining morphology parameters primarily control geometry and attachment. We therefore use two parameter-group ablations to better understand their contributions.

We have added Appx. I: “Morphology Parameter-Group Ablations” (Fig. 13), which reports:

We examine the contribution of different morphology parameter groups through two additional ablations. Fixed presence holds limb presence at its initial value while optimizing the remaining morphology parameters, whereas presence only optimizes presence while fixing the remaining morphology parameters. Across four matched seeds, full FORM achieves a mean best-so-far return of 4.55, compared with 0.97 and 0.41 for the two ablations, respectively (Fig. 13). These results highlight the benefit of jointly optimizing presence and the remaining morphology parameters.

Although these ablations cannot uniquely attribute the improvements to topology or geometry, they show that neither restricted variant approaches the performance of full FORM, supporting the benefit of their joint optimization.


Q2: FORM adaptively increases the center fraction and ends with an all-center phase, whereas CMA-ES uses a fixed fraction of 0.125. With the same fixed fraction, FORM reaches only 1.37 (Fig. 5a), below CMA-ES’s 2.11 (Fig. 3). . Could the authors discuss how much of FORM’s advantage comes from this schedule rather than from the pathwise gradients?

We apologize for the confusion. We realize now that our original submission did not clearly explain why we used a fixed center fraction for CMA-ES while FORM employed competence-based allocation. We have now clarified the motivation behind this design choice in Sect. 3.1.

Specifically, as stated in the revised manuscript:

Unlike FORM, which can continue optimizing morphology through pathwise gradients even when all environments use the same center design (c=1c=1), CMA-ES relies on evaluations of sampled candidate morphologies to update its search distribution. Without such evaluations, CMA-ES cannot continue optimizing morphology.

For a fair comparison that preserves CMA-ES's ability to optimize morphology throughout training, we therefore keep CMA-ES at FORM's initial center fraction of c=0.125c=0.125 in the main comparison.

In other words, our intention was to allow CMA-ES to continue searching for better morphologies throughout training. We recognize that this was not clear in the original submission.

Nevertheless, we agree with the reviewer that center allocation itself can affect performance. To address this concern directly, we have added Appx. G: Effect of Center-Allocation Schedules (Fig. 11), which compares both methods under identical allocation rules.

As reported in Appx. G:

Across eight matched seeds, at a common wall-clock cutoff of 8.318.31 hours (compared to 8.468.46 hours in Fig. 3), FORM achieves higher mean best-so-far returns under both settings: 4.014.01 versus 2.292.29 with fixed allocation, and 5.665.66 versus 2.782.78 with competence-based allocation.

We would also like to clarify the difference from the earlier fixed-allocation ablation. The previously reported mean of 1.37 was based on four seeds (1000–1003). The four additional seeds (1004–1007) achieved substantially higher returns, bringing the eight-seed mean to 4.01, with all original runs retained. This highlights the substantial seed-to-seed variability of fixed-allocation FORM.

Importantly, FORM achieves a higher mean return under both matched allocation schedules. Even fixed-allocation FORM (4.01) exceeds CMA-ES with competence-based allocation (2.78). These results suggest that while center allocation contributes to FORM's performance, it does not alone explain its higher mean return.

We hope these results clarify the contribution of center allocation to FORM's observed performance advantage.

I thank the authors for their detailed response and additional experiments. I believe that making morphology–control co-design differentiable is a valuable direction for improving optimization efficiency. However, I still have concerns about the comparisons with BodyGen and Transform2Act. These methods were originally designed around their own topology construction mechanisms, rather than continuous optimization within a fixed supergraph. These approaches represent different ways of performing morphology–control co-design, and removing the original topology construction mechanisms may not fully reflect the capabilities and efficiency of these baselines. Since sample efficiency is a key claimed advantage of FORM, I believe the current comparisons do not fully establish its efficiency advantage over the original methods. I will therefore maintain my original score.

Reviewer_7JNROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes FORM, a method for jointly co-optimizing the topology, geometry, and control of articulated rigid-body robots using first-order gradients from a differentiable rigid-body simulator with contact. A continuous relaxation of per-appendage "presence" over a fixed candidate kinematic supergraph lets a shared controller and the design distribution be updated jointly from gradients aggregated across sampled morphologies and short simulation horizons. On a flat-ground forward-locomotion task over 8 seeds, FORM outperforms a CMA-ES baseline under matched agent-step and wall-clock budgets, with ablations on the sampling/center-allocation schedule, gradient-aggregation scheme, and update interval, plus robustness checks via limb pruning and fresh-controller retraining.

Critical strengths

Prior differentiable co-design methods (Ma et al., 2021; Matthews et al., 2023; Cochevelou et al., 2023; Kobayashi et al., 2024) were restricted to soft, jointless, volumetrically actuated bodies, and other rigid-body approaches either fixed the topology in advance (Ha et al., 2017; Spielberg et al., 2017) or relied on non-differentiable evolutionary operators to change it (Strgar et al., 2024). FORM's continuous relaxation of appendage presence and attachment (Section 2.3) lets pathwise gradients flow through these otherwise discrete topological decisions inside a differentiable rigid-body simulator with contact, closing a genuine methodological gap in the literature.

Critical weaknesses

Weakness 1: The main comparison does not isolate the contribution of first-order morphology gradients, and the only genuine co-design baseline (CMA-ES) may be confounded by an additional algorithmic difference beyond the gradient estimator.

FORM and CMA-ES differ not only in how the morphology update is computed, but also in the center-allocation schedule. CMA-ES is run with a fixed center fraction of c=0.125 throughout training (Section 3.1), whereas FORM uses a competence-based schedule that raises c from 0.125 to 1 (Section 2.1; Algorithm 1). The paper's own ablation (Fig. 5a, Section 3.3) shows that fixing FORM's center fraction at 0.125 ("w/o center adjustment") drops the mean best-so-far return from 4.55 to 1.37, a drop of more than 3×. Because CMA-ES is evaluated under exactly this low, fixed allocation, it is unclear how much of FORM's reported advantage over CMA-ES (5.85 vs. 2.11, Section 3.2) is attributable to the first-order gradient estimator itself rather than to this schedule confound. A controlled, matched-seed comparison between CMA-ES and the fixed-center-allocation FORM variant (c=0.125) would help disentangle the contribution of first-order morphology gradients from that of the adaptive allocation schedule.

In addition, none of the zeroth-order or learned co-design methods discussed at length in the Introduction (Schaff et al., 2019; Yuan et al., 2022, "Transform2act"; Lu et al., 2025, "BodyGen") are included as empirical baselines. BodyGen in particular targets the same design/control reward-imbalance problem that motivates FORM, so its absence from the comparison makes it difficult to situate FORM's reported gains relative to the broader morphology co-design literature.

Weakness 2: The experimental evaluation is too narrow, both in task scope and in what is measured, to support the paper's claims about agile, graceful locomotion and the reliability of the discovered designs.

All quantitative results come from a single flat-ground task with reward rt=xt+1−xt (pure forward displacement; Section 3.1); there is no energy, smoothness, or robustness term, and no additional task or terrain is evaluated. Forward displacement is a reasonable proxy for raw speed, but the broader claims of "agile" and "move quickly and gracefully" (Abstract, Introduction) are not measured by any reported metric, such as gait smoothness, energy efficiency, or robustness to disturbances.

Three of the eight independent evolutionary trials did not surpass the fixed, hand-designed Ant-inspired baseline after the full training budget (Section 3.2: "Five of the eight FORM runs also exceed the fixed Ant reference"). This variation across seeds relative to the fixed-morphology reference is not analyzed or discussed, leaving open whether it reflects sensitivity to initialization, training instability, or other factors.

Although the paper includes ablations on center allocation and morphology-update intervals, it does not report sensitivity analyses for several other key hyperparameters. Most notably, the short-horizon differentiation length H=8 (Section 2.1; Table 2), which directly governs the bias-variance tradeoff of the pathwise morphology gradient, is fixed at a single value throughout. Likewise, the morphology learning-rate schedule and the sampling scale σ are each used with only one setting (Table 3). Without this analysis, it is difficult to know how robust FORM's reported gains are to these choices.

Questions for authors
  1. Could the authors report a matched-seed comparison between CMA-ES and the fixed-center-allocation FORM variant (c=0.125), to better isolate the contribution of first-order morphology gradients from the allocation schedule?

  2. Could the authors include a comparison against at least one of the zeroth-order co-design methods discussed in the Introduction (e.g., BodyGen, Transform2act)?

  3. What explains the three seeds (out of eight) that did not surpass the fixed Ant-inspired baseline? Is there a common cause across these runs?

  4. How sensitive is FORM's performance to the differentiation horizon H and to the morphology sampling scale σ? Were these values tuned, and if so, on what criterion?

  5. Could the authors report a metric beyond cumulative forward displacement (e.g., energy use, smoothness, or robustness to perturbation) to substantiate the claims of "agile" and "graceful" locomotion?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 7JNR (Part 1/3)

We thank the reviewer for their careful examination of our work. We have conducted several additional experiments to address the reviewer’s questions and have revised the manuscript accordingly.


Weakness 1 (1) : The main comparison does not isolate the contribution of first-order morphology gradients, and the only genuine co-design baseline (CMA-ES) may be confounded by an additional algorithmic difference beyond the gradient estimator. FORM and CMA-ES differ not only in how the morphology update is computed, but also in the center-allocation schedule. CMA-ES is run with a fixed center fraction of c=0.125 throughout training (Section 3.1), whereas FORM uses a competence-based schedule that raises c from 0.125 to 1 (Section 2.1; Algorithm 1). The paper's own ablation (Fig. 5a, Section 3.3) shows that fixing FORM's center fraction at 0.125 ("w/o center adjustment") drops the mean best-so-far return from 4.55 to 1.37, a drop of more than 3×. Because CMA-ES is evaluated under exactly this low, fixed allocation, it is unclear how much of FORM's reported advantage over CMA-ES (5.85 vs. 2.11, Section 3.2) is attributable to the first-order gradient estimator itself rather than to this schedule confound. A controlled, matched-seed comparison between CMA-ES and the fixed-center-allocation FORM variant (c=0.125) would help disentangle the contribution of first-order morphology gradients from that of the adaptive allocation schedule.

We apologize for the confusion. Our original manuscript did not clearly explain why we used a fixed center fraction for CMA-ES while FORM employed competence-based allocation. We have now clarified the motivation behind this design choice in Sect. 3.1.

Specifically, as stated in the revised manuscript:

Unlike FORM, which can continue optimizing morphology through pathwise gradients even when all environments use the same center design (c=1c=1), CMA-ES relies on evaluations of sampled candidate morphologies to update its search distribution. Without such evaluations, CMA-ES cannot continue optimizing morphology.

For a fair comparison that preserves CMA-ES's ability to optimize morphology throughout training, we therefore keep CMA-ES at FORM's initial center fraction of c=0.125c=0.125 in the main comparison.

In other words, our intention was to allow CMA-ES to continue searching for better morphologies throughout training. We recognize that this was not clear in the original submission.

Nevertheless, we agree with the reviewer that center allocation itself can affect performance. To address this concern directly, we have added Appx. G: Effect of Center-Allocation Schedules (Fig. 11), which compares both methods under identical allocation rules.

As reported in the new appendix:

Across eight matched seeds, at a common wall-clock cutoff of 8.318.31 hours (compared to 8.468.46 hours in Fig. 3), FORM achieves higher mean best-so-far returns under both settings: 4.01 versus 2.29 with fixed allocation, and 5.66 versus 2.78 with competence-based allocation.

We would also like to clarify the difference from the earlier fixed-allocation ablation. The previously reported mean of 1.37 was based on four seeds (1000–1003). The four additional seeds (1004–1007) achieved substantially higher returns, bringing the eight-seed mean to 4.01, with all original runs retained. This highlights the substantial seed-to-seed variability of fixed-allocation FORM.

Importantly, FORM achieves a higher mean return under both matched allocation schedules. Even fixed-allocation FORM (4.01) exceeds CMA-ES with competence-based allocation (2.78). These results suggest that while center allocation contributes to FORM's performance, it does not alone explain its higher mean return.

We hope that the clarified motivation in Sect. 3.1 and the additional matched comparisons in Appx. G address the reviewer's concern.

AuthorsOpenReview ↗

Response to Reviewer 7JNR (Part 2/3)

Weakness 1 (2) : In addition, none of the zeroth-order or learned co-design methods discussed at length in the Introduction (Schaff et al., 2019; Yuan et al., 2022, "Transform2act"; Lu et al., 2025, "BodyGen") are included as empirical baselines. BodyGen in particular targets the same design/control reward-imbalance problem that motivates FORM, so its absence from the comparison makes it difficult to situate FORM's reported gains relative to the broader morphology co-design literature.

We thank the reviewer for suggesting these comparisons. We now compare against BodyGen and Transform2Act across eight matched initialization seeds (Sects. 3.1–3.2, Figs. 3 and 5).

As described in the revised Sect. 3.1:

We adapt BodyGen and Transform2Act to the same candidate supergraph and simulation environment, retaining their respective policy architectures, PPO objectives, and credit-assignment mechanisms while omitting discrete topology construction.

We acknowledge that these adaptations omit the original methods' discrete topology-editing ability and therefore compare optimization within a shared candidate supergraph.

The revised Sect. 3.2 reports:

At the matched wall-clock budget of 8.46 hours, FORM reaches a mean best-so-far return of 5.77 ± 0.85, compared with 2.32 ± 0.20 for CMA-ES, 2.13 ± 0.35 for BodyGen, and 2.29 ± 0.31 for Transform2Act (mean ± s.e. across eight seeds).

FORM achieves higher returns in all eight matched runs against each baseline (Fig. 3C; (p=0.0078) for each paired comparison). Fig. 5 further compares the selected morphologies.

We also explicitly acknowledge that:

Despite its higher mean performance, FORM exhibits greater variability across seeds than the baselines.


Weakness 2 (1) : The experimental evaluation is too narrow, both in task scope and in what is measured, to support the paper's claims about agile, graceful locomotion and the reliability of the discovered designs.

All quantitative results come from a single flat-ground task with reward rt=xt+1−xt (pure forward displacement; Section 3.1); there is no energy, smoothness, or robustness term, and no additional task or terrain is evaluated. Forward displacement is a reasonable proxy for raw speed, but the broader claims of "agile" and "move quickly and gracefully" (Abstract, Introduction) are not measured by any reported metric, such as gait smoothness, energy efficiency, or robustness to disturbances.

We agree that forward displacement alone does not establish graceful locomotion. However it can be a good proxy. Yu et al. (2026) equate agility with nonquasistatic behavior. Beyond the bright line of quasistatic versus nonquasistatic, we find terms like quick, strong, agile and graceful to be helpful albeit relative, qualitative descriptions. Here we are comparing against the relatively slow and weak, soft robots that were evolved in prior work using gradient information. As we described in our Introduction, these were clearly neither agile nor graceful. They visibly struggled to move on land and often dragged parts of their body along the surface as they inched forward.

In addition, to address the reviewer's request for a robustness evaluation, we have added Appx. E: Independent Performance Evaluation and Friction Sensitivity (Fig. 9B). Specifically, we evaluated the eight selected morphology–controller pairs under friction multipliers of 0.8, 1.0, and 1.2, using 64 independent stochastic rollouts per design at each setting. Mean returns remained largely stable across all eight designs, suggesting that their locomotion is not narrowly tuned to the training friction.

References:

Yu et al. (2026), PNAS. https://www.pnas.org/doi/10.1073/pnas.2519129123

AuthorsOpenReview ↗

Response to Reviewer 7JNR (Part 3/3)

Weakness 2 (2) : Three of the eight independent evolutionary trials did not surpass the fixed, hand-designed Ant-inspired baseline after the full training budget (Section 3.2: "Five of the eight FORM runs also exceed the fixed Ant reference"). This variation across seeds relative to the fixed-morphology reference is not analyzed or discussed, leaving open whether it reflects sensitivity to initialization, training instability, or other factors.

To investigate performance variation across evolutionary trials, we have added Appx. J: “Continued controller training on fixed morphologies” (Fig. 14), in which we freeze the learned morphologies and continue training their original controllers for 128 additional episodes.

Of the three runs that initially fell below the fixed Ant baseline, two reached the Ant reference after additional controller training, suggesting that incomplete controller adaptation may have contributed to their lower performance.

The remaining run stayed below Ant, but a separate co-design run with the same training seed and twice the morphology sampling scale achieved a best-so-far return of 5.42 instead of 2.50, suggesting sensitivity to morphology sampling.

These observations suggest that both controller adaptation and morphology optimization may contribute to the variation across seeds.


Weakness 2 (3) : Although the paper includes ablations on center allocation and morphology-update intervals, it does not report sensitivity analyses for several other key hyperparameters. Most notably, the short-horizon differentiation length H=8 (Section 2.1; Table 2), which directly governs the bias-variance tradeoff of the pathwise morphology gradient, is fixed at a single value throughout. Likewise, the morphology learning-rate schedule and the sampling scale σ are each used with only one setting (Table 3). Without this analysis, it is difficult to know how robust FORM's reported gains are to these choices.

We have added a more comprehensive hyperparameter sensitivity analysis to Appx. H: “Hyperparameter sensitivity” (Fig. 12), which examines the horizon length, morphology sampling scale, and morphology learning rate across four matched seeds.

Our default hyperparameters were selected through limited preliminary experiments early in the project. We retained them during subsequent method development as these settings provided satisfactory performance.

For convenience, we summarize the key findings from the revised manuscript:

The default H=8H=8 and shorter H=4H=4 achieve similar mean returns, whereas H=16H=16 substantially reduces performance. The default sampling scale σ0\sigma_0 outperforms both 0.5σ00.5\sigma_0 and 2σ02\sigma_0. Halving the morphology learning rate reduces performance, whereas doubling it yields a slightly higher mean return.

Notably, at H=16H=16, gradient clipping occurs in 22.56% of updates, compared with 0.1% at H=8H=8, suggesting possible optimization instability at longer differentiation horizons.


Q1: Could the authors report a matched-seed comparison between CMA-ES and the fixed-center-allocation FORM variant (c=0.125c=0.125), to better isolate the contribution of first-order morphology gradients from the allocation schedule?

Please see our response above to W1 (1)

Q2: Could the authors include a comparison against at least one of the zeroth-order co-design methods discussed in the Introduction (e.g., BodyGen, Transform2act)?

Please see our response above to W1 (2)

Q3: What explains the three seeds (out of eight) that did not surpass the fixed Ant-inspired baseline? Is there a common cause across these runs?

Please see our response above to W2 (2)

Q4: How sensitive is FORM's performance to the differentiation horizon H and to the morphology sampling scale σ? Were these values tuned, and if so, on what criterion?

Please see our response above to W2 (3)

Q5: Could the authors report a metric beyond cumulative forward displacement (e.g., energy use, smoothness, or robustness to perturbation) to substantiate the claims of "agile" and "graceful" locomotion?

Please see our response above to W2 (1)

Reviewer_9SQMOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents FORM, a method for jointly optimizing robot morphology and control through differentiable rigid-body simulation. It continuously adjusts appendage geometry, attachment, and presence within a candidate kinematic supergraph. Short-horizon gradients, local morphology sampling, and a shared controller allow initially immotile capsule-shaped agents to develop articulated bodies with different locomotion behaviors.

Critical strengths

The paper extends gradient-based morphology optimization to articulated rigid-body agents. Combining a continuous morphology representation, differentiable contact simulation, and short-horizon optimization into a working co-design system is a useful contribution. The emergence of distinct bodies capable of effective locomotion from the initial capsule provides compelling evidence of what the approach can achieve.

The experiments provide good support for the method. The CMA-ES comparison considers both interaction and wall-clock budgets with matched initializations. Beyond component ablations, the authors prune low-presence appendages and retrain controllers from scratch on the resulting bodies. These checks help establish that the learned designs do not depend on vestigial parts or a particular co-adapted controller.

Critical weaknesses

The scope of “topology evolution” should be stated more precisely. All appendages already exist in a fixed candidate supergraph; optimization changes their deployment, size, and contribution, while actual removal happens afterward. This is a reasonable relaxation, but differs from discovering new connectivity. Since the experiments use only one supergraph, it is also unclear how sensitive the results are to the number of candidate appendages and their initial internal arrangement.

The reliability of the morphology gradients is not directly evaluated. State gradients are truncated at short-horizon boundaries, and a learned value function estimates the remaining return. The resulting update is therefore not the exact design gradient of a full rollout. The ablations support aggregating gradients across horizons, but do not show whether those gradients predict the benefit of actual morphology changes. Comparing gradient predictions with returns under small design perturbations would strengthen the methodological evidence.

The evaluation remains limited to forward locomotion on flat ground under one contact and actuator model. The main curves also report best-so-far training returns, which establish what the search encountered but provide less information about repeatable performance. Physical hardware is not necessary to address this concern: independent evaluations and modest changes in friction, contact stiffness, or terrain would help determine whether the designs remain effective beyond the precise optimization setting.

Questions for authors
  1. How sensitive is FORM to the number of candidate appendages and their initial internal arrangement? Do these choices substantially affect the final number of functional limbs or the resulting performance?

  2. Can the authors compare morphology-gradient predictions with measured return changes under small design perturbations at several stages of training?

  3. After selecting the final designs, how consistent is their performance across independent rollouts and modest changes in physical parameters? What failure modes appear?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 9SQM (Part 1/2)

We appreciate the reviewer’s thoughtful questions, constructive suggestions, and positive assessment of our contributions. We have conducted several additional experiments to address the reviewer’s comments and have revised the manuscript accordingly.


W1: The scope of “topology evolution” should be stated more precisely. All appendages already exist in a fixed candidate supergraph; optimization changes their deployment, size, and contribution, while actual removal happens afterward. This is a reasonable relaxation, but differs from discovering new connectivity. Since the experiments use only one supergraph, it is also unclear how sensitive the results are to the number of candidate appendages and their initial internal arrangement.

We understand that the word “topology” means different things to different communities. To avoid ambiguity, in our revision, we have added the following clarification (new text italicized):

FORM leverages short-horizon differentiation and represents structural variation through continuous variables over a candidate kinematic supergraph, thereby preserving pathwise morphology gradients without presupposing the agent’s final topology, i.e. the number and distribution of limbs with which it pushes and pulls against the world

Here, final topology refers to the robot’s effective kinematic tree after evolution. Although a supergraph is indeed predefined, the final topology of limbs is not. When a link is small or shrinks, its motor strength is strongly attenuated, making it effectively disappear from the functional body. This is supported by our pruning experiment (Sect. 3.4, Fig. 7), where removing very small limb buds, without retraining the controller, leaves locomotion performance virtually unchanged.

To investigate supergraph sensitivity, we have added Appx. K: Alternative Candidate Supergraphs (Figs. 15–18), exploring four-, six-, and eight-appendage supergraphs, as well as a hierarchical appendage-on-appendage configuration.

Using four common seeds and identical training budgets, we report:

Their mean best-so-far returns are 4.98, 4.55, and 5.15, respectively, showing no consistent trend with candidate count across the tested configurations. Despite different supergraph sizes, 10 of the 12 selected designs retain between three and five appendages after presence-based pruning.

These results suggest that FORM is relatively insensitive to candidate appendage count under the random initializations tested.

We have also clarified the scope of topology evolution in Appx. K:

Within each run, the candidate connectivity remains fixed, while FORM continuously optimizes appendage presence, geometry, and attachment locations.


W2: The reliability of the morphology gradients is not directly evaluated. State gradients are truncated at short-horizon boundaries, and a learned value function estimates the remaining return. The resulting update is therefore not the exact design gradient of a full rollout. The ablations support aggregating gradients across horizons, but do not show whether those gradients predict the benefit of actual morphology changes. Comparing gradient predictions with returns under small design perturbations would strengthen the methodological evidence.

To address this weakness, we have added Appx. F: “Effect of morphology-gradient updates” (Fig. 10), to directly evaluate whether small gradient-directed morphology perturbations improve full-episode performance at different stages of training.

For convenience, we quote the key findings from the revised manuscript:

At 25%, 50%, and 75% of training, the mean paired return changes are −0.0002, +0.0252, and +0.0393, respectively. These results suggest that small gradient-based morphology updates can improve full-episode performance during mid-to-late training, with no clear improvement early in training.

Although these results do not guarantee improvement at every training point, they provide direct evidence that FORM's gradient-based updates can identify beneficial morphology changes.

AuthorsOpenReview ↗

Response to Reviewer 9SQM (Part 2/2)

W3: The evaluation remains limited to forward locomotion on flat ground under one contact and actuator model. The main curves also report best-so-far training returns, which establish what the search encountered but provide less information about repeatable performance. Physical hardware is not necessary to address this concern: independent evaluations and modest changes in friction, contact stiffness, or terrain would help determine whether the designs remain effective beyond the precise optimization setting.

To address this concern, we have added Appx. E: “Independent performance evaluation and friction sensitivity” (Fig. 9) to assess FORM’s repeatability and robustness. Specifically, we re-evaluated the selected morphology–controller pairs using independent stochastic rollouts and tested their sensitivity to moderate friction variations.

For convenience, we quote the key findings from the revised manuscript:

The resulting return distributions are generally close to their selected training scores, indicating that the learned performance is reproducible across independent rollouts. Occasional low-return outliers occur when robots topple and fail to recover effective locomotion.

Mean returns vary only modestly across these friction settings, suggesting that the learned agents maintain effective locomotion beyond the exact friction setting used during training.

We also noticed our original wording could have suggested that we selected the maximum return based on individual rollouts. In fact, we compute the cumulative maximum of center-cohort mean returns, with at least eight environments per cohort. This makes the reported training metric less sensitive to isolated high-return rollouts. We have also clarified the definition of best-so-far training return in the revised manuscript (Sect. 3.1):

For FORM and CMA-ES, for each seed, we compute the best-so-far return as the cumulative maximum of the center-cohort mean episode returns.

We then report the mean and standard error of the best-so-far return across seeds.


Q1: How sensitive is FORM to the number of candidate appendages and their initial internal arrangement? Do these choices substantially affect the final number of functional limbs or the resulting performance?

Please see our response to W1.

Q2: Can the authors compare morphology-gradient predictions with measured return changes under small design perturbations at several stages of training?

Please see our response to W2.

Q3: After selecting the final designs, how consistent is their performance across independent rollouts and modest changes in physical parameters? What failure modes appear?

Please see our response to W3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We sincerely thank all reviewers for their careful assessments and constructive feedback. We particularly appreciate their recognition of FORM's central contribution: extending gradient-based morphology optimization to articulated rigid-body agents. Reviewer 9SQM described this extension as a useful contribution, w5vv found the work well motivated, and 7JNR noted that it closes a genuine methodological gap. We also appreciate their positive assessments of our experiments, ablations, and analyses (9SQM, w5vv).

The reviewers' suggestions have helped us substantially improve the manuscript through additional evaluations and clearer discussion of FORM's scope and limitations. We summarize the main revisions below. Detailed responses can be found in the individual reviewer threads.


1. Controlled CMA-ES comparisons (7JNR W1(1)/Q1; w5vv Q2)

Appx. G (Fig. 11) compares FORM and CMA-ES under matched fixed and competence-based center-allocation schedules across eight matched seeds. FORM achieves higher mean best-so-far returns under both schedules (4.01 vs. 2.29; 5.66 vs. 2.78), suggesting that center allocation alone does not explain FORM's advantage.

2. Additional co-design baselines (7JNR W1(2)/Q2; w5vv W2 and follow-up)

We added fixed-supergraph adaptations of BodyGen and Transform2Act (Sect. 3.1–3.2; Figs. 3, 5), where FORM achieves higher mean returns under matched wall-clock budgets. We also compared with baselines retaining discrete topology construction mechanisms (Appx. M, Fig. 20; Table 2). The strongest discrete-topology baseline achieved a mean best-so-far return of 3.99 after 20.48M agent steps, versus FORM's 5.85 after 5.57M, supporting FORM's sample efficiency in our task, while we do not claim universal superiority over the original methods in their native settings.

3. Structural flexibility and environmental scope (9SQM W1/Q1, W3; w5vv W1/W3)

We clarified that FORM continuously optimizes appendage presence, geometry, and attachment within a configurable candidate supergraph, whose connectivity remains fixed within each run. This allows diverse effective morphologies to emerge without discrete graph-editing operations. Appx. K (Figs. 15–18) demonstrates this flexibility with four- and eight-appendage supergraphs and hierarchical structures, beyond the original six-appendage configuration. Appx. L (Fig. 19) provides preliminary stair-climbing results, where FORM achieves higher mean returns than CMA-ES.

4. Robustness and seed variability (9SQM W3/Q3; 7JNR W2(1–2)/Q3/Q5)

Appx. E (Fig. 9) evaluates independent rollouts and friction variations, showing reproducible performance and stable mean returns across the tested friction settings. Appx. J (Fig. 14) shows that two of three initially below-Ant runs reached the reference after further controller training, suggesting that incomplete controller adaptation may contribute to seed-level variability.

5. Hyperparameters and morphology components analysis (7JNR W2(3)/Q4; w5vv Q1)

Appx. H (Fig. 12) examines horizon length, morphology sampling scale, and learning rate. Although topology and geometry cannot be fully disentangled, Appx. I (Fig. 13) shows that fixed-presence and presence-only variants both underperform joint morphology optimization.

6. Morphology-gradient reliability (9SQM W2/Q2)

Appx. F (Fig. 10) measures full-episode return changes following small gradient-directed morphology perturbations, finding positive mean improvements during mid-to-late training, which shows FORM's gradient-based updates can identify beneficial morphology changes.


Together, these additional experiments and analyses strengthen the evidence for FORM's effectiveness in differentiable morphology–control co-design of articulated rigid-body robots, while clarifying its scope and limitations. We sincerely thank the reviewers again for their thoughtful feedback and constructive discussions, which has helped us substantially improve the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.