acceptodds
Under review as a conference paper at ICLR 2027

Evolving Dexterous Robots from Scratch

Abstract

Little is known about how to manually design agents capable of dexterous manipulation. Some design principles have been inferred from close examination of how animals manipulate objects, but these structures and behaviors have so far resisted biomimicry and may not be optimal for artificial machines. Here we evolve freeform robots to pick up, hold, rotate, and use diverse objects. Unlike other approaches to optimizing robot hands, we do not presuppose the presence, articulation, or geometry of any part of the body. Although familiar prehensile forms such as tails, beaks, paws and claws may emerge spontaneously under certain conditions—and while such conditions could be of interest to evolutionary biologists—de novo manipulator design can also reveal whole new solutions, overlooked or unknown structures which may be better suited for the task at hand. We use contrastive learning to create a highly searchable genetic embedding of design space, an autoregressive developmental model to decode designs, evolutionary strategies to find good designs, and reinforcement learning to train each evolved design. Winning designs were automatically converted into a manufacturable blueprint, printed, assembled and tested in the real world in a zero-shot manner. The results represent the state-of-the-art in evolutionary robotics in terms of performance, diversity and complexity. Project page: https://evodexbots.github.io

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotics, morphology, evolution, manipulation, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_fSqKOpenReview ↗
Rating
2(−1)
Confidence
3(−1)
Informativeness
1(−1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an end-to-end framework for evolving dexterous robotic manipulators from scratch, without imposing predefined hand structures. The authors define a flexible morphology space consisting of a root body and up to five appendages, and generate two million synthetic morphologies to train a contrastive encoder and an autoregressive decoder. The resulting latent representation enables morphology optimization using CMA-ES, while a PPO policy is independently trained to evaluate each candidate design.

The framework is evaluated on six dexterous manipulation tasks, including object rotation, picking, hanging, retrieval, flipping, and holding. The authors additionally investigate the evolution of a generalist manipulator capable of performing multiple tasks. Finally, selected evolved designs are manufactured using 3D-printed components and servos, with their corresponding policies distilled into proprioceptive controllers and fine-tuned with domain randomization for zero-shot sim-to-real deployment.

Critical strengths
  • Comprehensive end-to-end pipeline for dexterous manipulator evolution: The paper presents a comprehensive framework that spans morphology space construction, latent representation learning, evolutionary optimization, policy learning, and real-world deployment. Unlike much of the prior work on morphology evolution, which primarily focuses on locomotion or relies on predefined manipulator structures, this work demonstrates the feasibility of evolving diverse dexterous manipulators from scratch. The successful fabrication and real-world deployment of evolved designs further substantiate the practical value of the proposed framework. I consider this a substantial contribution to evolutionary robotics.

  • Convincing demonstrations of diverse evolved morphologies and their practical effectiveness: The experiments demonstrate that the framework can discover substantially different morphologies adapted to diverse manipulation tasks. Particularly compelling are the real-world demonstrations and the improvements over existing manipulator designs in object rotation. These results provide meaningful evidence that automated morphology evolution can discover effective manipulation strategies beyond conventional, manually designed structures.

Critical weaknesses
  • Limited investigation of evolutionary optimization algorithms: Although the paper proposes a comprehensive morphology evolution framework, the evolutionary optimization itself relies exclusively on CMA-ES. Alternative approaches, such as cluster-based evolutionary optimization [1], could potentially provide better exploration of the diverse morphology space. Since discovering novel and diverse manipulator designs is a central objective of the paper, a comparison with alternative evolutionary strategies would help establish whether the reported improvements arise primarily from the proposed latent representation or depend on the choice of evolutionary optimizer.

  • Limited training scale in the latent representation ablation: The authors train their proposed representation using two million synthetic morphologies but conduct the representation-learning ablations using only 200,000 samples. Although this provides a controlled comparison under a reduced data budget, it remains unclear whether the observed advantages over alternative representations, particularly the VAE baseline, persist at the full training scale. Since the quality of the latent representation is central to the paper's contributions, evaluating the baselines under the same two-million-sample training budget would provide stronger evidence for the effectiveness of the proposed representation.

[1] Hyeonseong Jeon et al. Convergent Functions, Divergent Forms. NeurIPS 2025.

Questions for authors
  • Q1. Multi-task versus task-specific morphology optimization: Have the authors quantitatively compared the performance of the evolved multi-task manipulator against the task-specific specialists on each individual task? Section 3.8 provides a qualitative discussion of their relative performance, but a quantitative comparison would help clarify the trade-off between generality and specialization. More importantly, could the authors compare the evolved multi-task morphology against a conventional dexterous manipulator trained with a sufficiently capable multi-task policy? Such a comparison would help establish whether morphology evolution provides benefits beyond those achievable through policy optimization alone.

  • Q2. Alternative latent representation learning approaches: Have the authors considered combining contrastive learning with a variational autoencoder, rather than relying on either contrastive reconstruction learning or a conventional VAE? A contrastive VAE could potentially preserve the reconstruction properties of a VAE while encouraging morphological similarity in latent space. Given that the proposed representation is central to the framework, such a comparison could further clarify which representation-learning properties are most important for evolvability.

9 comments

We thank the reviewer for their time and thoughtful assessment. We appreciate that the reviewer found our work to be convincing, compelling, and a substantial contribution to evolutionary robotics.

Below we address the stated weaknesses (W1-W2), questions (Q1-Q3), and ethics concern (E1), one by one, also referencing how our manuscript was modified in response. The updated manuscript has been uploaded to OpenReview with additions marked in red text. We believe all changes have significantly improved the paper, and we thank the reviewer for prompting them.

 

W1. Limited investigation of evolutionary optimization algorithms: Although the paper proposes a comprehensive morphology evolution framework, the evolutionary optimization itself relies exclusively on CMA-ES. Alternative approaches, such as cluster-based evolutionary optimization [1], could potentially provide better exploration of the diverse morphology space. Since discovering novel and diverse manipulator designs is a central objective of the paper, a comparison with alternative evolutionary strategies would help establish whether the reported improvements arise primarily from the proposed latent representation or depend on the choice of evolutionary optimizer.

[1] Hyeonseong Jeon et al. Convergent Functions, Divergent Forms. NeurIPS 2025.

We agree that our paper relies exclusively on CMA-ES and that other algorithms could provide better exploration of morphospace. With respect, we feel that an investigation of different evolutionary algorithms is outside the scope of this paper. However, to acknowledge this limitation, we have added the following text and references to Discussion:

“In this paper, however, morphological optimization was driven somewhat myopically for task performance. In order to isolate the effect of the proposed genetic representation, and for consistency with prior work, the same search algorithm (CMA-ES) was used across all evolutionary trials. Morphological diversity was largely a byproduct of evolving a distribution of designs, rather than a single design, and starting each trial from different random initial conditions. While the sensitivity of our results to the choice of search algorithm remains to be determined, it seems likely that exploration would be increased by introducing additional selection pressures, such as those that localize competition to promote speciation (Lehman & Stanley, 2011; Jeon et al., 2026).”

References:
Lehman and Stanley (2011) GECCO. https://dl.acm.org/doi/10.1145/2001576.2001606
Jeon et al. (2025) NeurIPS. https://arxiv.org/abs/2505.21665

 

W2. Limited training scale in the latent representation ablation: The authors train their proposed representation using two million synthetic morphologies but conduct the representation-learning ablations using only 200,000 samples. Although this provides a controlled comparison under a reduced data budget, it remains unclear whether the observed advantages over alternative representations, particularly the VAE baseline, persist at the full training scale. Since the quality of the latent representation is central to the paper's contributions, evaluating the baselines under the same two-million-sample training budget would provide stronger evidence for the effectiveness of the proposed representation.

We have followed the reviewer’s suggestion and retrained all representation-learning models on the full dataset of 2M example body plans. We are currently rerunning five independent evolutionary trials with each model. Once these trials are complete, we will replace the 200k ablation results with the full-scale comparisons, so all models are evaluated under the same representation-training and evolutionary evaluation budgets. Each evolutionary trial requires approximately 33 hours on four H100 GPUs, so these experiments are still in progress. We will update the manuscript and this response with the results and statistical analysis before the end of the rebuttal period.

Q1. Multi-task versus task-specific morphology optimization: Have the authors quantitatively compared the performance of the evolved multi-task manipulator against the task-specific specialists on each individual task? Section 3.8 provides a qualitative discussion of their relative performance, but a quantitative comparison would help clarify the trade-off between generality and specialization. More importantly, could the authors compare the evolved multi-task morphology against a conventional dexterous manipulator trained with a sufficiently capable multi-task policy? Such a comparison would help establish whether morphology evolution provides benefits beyond those achievable through policy optimization alone.

We have added these quantitative comparisons to Table 11 in Appx. C.7, along with the following new text:

“To isolate the effect of morphological evolution, we trained the LEAP Hand (Shaw et al., 2023) on the same five tasks using the same reward functions, actuator torque limit, and policy-training budget. To do so, we trained an expert policy for each task and then distilled the five experts into a single task-conditioned policy. The evolved multitasker achieved higher fitness than the LEAP Hand on all tasks except for holding the hammer. See Table. 11.”

 

Q2. Alternative latent representation learning approaches: Have the authors considered combining contrastive learning with a variational autoencoder, rather than relying on either contrastive reconstruction learning or a conventional VAE? A contrastive VAE could potentially preserve the reconstruction properties of a VAE while encouraging morphological similarity in latent space. Given that the proposed representation is central to the framework, such a comparison could further clarify which representation-learning properties are most important for evolvability.

Good question. To answer it, we trained a contrastive VAE on the full 2M dataset and added the results to Table 13. Appx. D.3 now describes its implementation and results. We have copied and pasted the relevant text below for convenience.

“We also explored an alternative representation that combines contrastive learning with a VAE. This variant, VAE + Lcon (2M), adds our NT-Xent loss to the VAE objective and uses a free-bits threshold of ϵ = 0.05. The contrastive loss is applied to the L2-normalized posterior mean, with a weight of 1 and temperature of 0.5. As shown in Table 13, this variant achieved a Recall@1 of 0.92 and length and volume neighborhood correlations comparable to our model. However, joint-count reconstruction remained poor, with Joint acc. of 0.01. The contrastive loss improved the encoder’s representation, but the decoder still failed to reconstruct diverse designs accurately.”

 

For convenience, we have copied and pasted Tables 11 and 13 below:

 

Table 11: Multitask comparison. Raw fitness on each task for the best evolved morphology on that task, the evolved multitasker, and the LEAP Hand. The best (bold) and second best (italic) values in each column are marked.

MorphologyRotatePickHangFlipHold
Evolved specialists (best per task)2.9743.7217.8517.7499.80
Evolved multitasker0.2833.9715.0413.830.12
LEAP Hand with multi-task policy0.2521.962.850.0422.62

 

Table 13: VAE variants of the latent genome.

ModelApp. acc.Joint acc.ValidRecall@1Edit ρTopology 10-NN acc.Length 10-NN rVolume 10-NN r
ValiditySmoothness
VAE (200k)0.440.060.330.010.93 ± 0.130.020.790.79
VAE (2M)0.440.060.990.000.98 ± 0.070.080.600.54
VAE (2M, ϵ\epsilon = 0.05)0.440.140.950.020.98 ± 0.080.080.780.72
VAE + Lcon\mathcal{L}_{\mathrm{con}} (2M)0.440.010.890.920.97 ± 0.100.000.960.88

E1. Details Of Ethics Concerns: The paper presents a framework for automatically evolving dexterous robotic manipulators with minimal structural constraints. While this is a valuable contribution, the increasing capabilities of such manipulators may raise potential ethical concerns, particularly regarding dual-use applications and real-world deployment safety. I encourage the authors to include a brief discussion of these ethical implications and potential risks, which would help contextualize the broader societal impact of their work.

We thank the reviewer for encouraging us to consider the broader implications of our work. We have, to the best of our abilities, added a brief discussion attempting to contextualize the potential societal impact of this research, including a unique moral consideration. For convenience, we have copied and pasted the new text below:

“A major goal of this research program is to explore the space of possible robots beyond familiar forms. In practical terms, the ability to generate diverse morphologies that interact with the environment in different ways could provide better, cheaper, or safer options. It could also enrich our understanding of how the body simultaneously constrains and enables intelligence. An interesting ethical question is whether it is easier to align the behavior of robots with human form to human values. Or could other, more alignable forms evolve?”

Reviewer_YumhOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a framework for evolving freeform robotic manipulators. Each design is described by a root and up to five articulated appendages, voxelised, and embedded into a 32-dimensional latent space on a hypersphere. The embedding is trained with a contrastive objective over a voxel encoder and a kinematic-graph encoder. A grammar-masked autoregressive transformer decodes latent vectors back into design tokens. CMA-ES searches the latent space, and a PPO policy is trained from scratch for every candidate. Selected designs are converted into printable blueprints; their policies are distilled into proprioceptive students, fine-tuned under domain randomisation, and deployed zero-shot. Experiments cover six tasks (rotate, pick, hang, retrieve, flip, hold) and a multitask setting, and five designs are fabricated. The paper reports faster in-hand rotation than House of Dextra (Fay et al., 2026) and an anthropomorphic LEAP baseline. It also reports an ablation in which the contrastive genome improves evolvability over alternatives that include a VAE.

Critical strengths

The paper addresses an interesting research question to understand what manipulators emerge when the body plan is not fixed in advance. Six task environments are designed, and five physically distinct robots are fabricated and deployed without real-world fine-tuning. To my knowledge, using a contrastive objective to build a searchable latent genome for robot morphology is new. The design space is larger and less constrained than that of recent dexterous-hand co-design work.

Critical weaknesses

1. The novel contribution is much narrower than claimed, and important prior work is not cited.

Most components of the pipeline have direct precedent. Li et al. (2025) evolved freeform robots with CMA-ES in a learned latent space. Guo et al. (2026) converted such designs into fabrication-ready blueprints and deployed an evolved robot zero-shot. House of Dextra (Fay et al., 2026) co-designed, fabricated and deployed rigid, servo-driven hands for in-hand rotation, grasping and flipping with the same modular hardware infrastructure, going from optimised design to printed hardware directly. Per-design PPO, teacher–student distillation and automatic domain randomisation are standard. What remains new is the contrastive training objective for the latent genome; the voxel and kinematic-graph encoders it combines are themselves standard. The other new element is the application of freeform latent-space evolution to the dexterous-manipulation setting previously addressed with modular hands. This is a legitimate contribution, but a narrower one than the abstract and TL;DR convey.

Several claims also overlook relevant work. The introduction states that "the majority [of approaches for automatic robot design] focused on locomotor performance and completely ignored object manipulation", and the TL;DR claims to show "for the first time de novo manipulator design". However, the paper omits the co-design work that does address manipulation. Evolution Gym (Bhatia et al., 2021) co-designs voxel robots for carrying, catching and lifting objects. DERL (Gupta et al., 2021) evolves morphologies with per-agent PPO and evaluates them on manipulation tasks. DiffuseBot (Wang et al., 2023) generates and fabricates a soft gripper. Pan et al. (2021) obtain hand morphologies by optimising grasps over learned latent representations. Most closely, Ikemura et al. (2026, arXiv:2602.17921) co-design manipulation end-effectors with a pipeline structurally similar to this paper's:

  • CMA-ES searches a latent design space, there obtained by PCA over diffeomorphic deformations of curated meshes;
  • a privileged controller is distilled into a student policy that uses only deployable observations;
  • the optimised designs are 3D-printed and deployed zero-shot on a real robot.

Their latent space is linear and their end-effectors attach to a parallel-jaw gripper, so the work does not anticipate the contrastive genome or freeform articulated bodies. It does show that latent-space morphology search for manipulation, with distillation and zero-shot fabricated deployment, predates this submission. None of these works is cited. Grammar-constrained generation of valid designs, one step at a time, appears in RoboGrammar (Zhao et al., 2020) and N-LIMB (Schaff and Walter, 2022). GLSO (Hu et al., 2023) is cited, but is characterised as limited to "stick figures". Encoding a robot's kinematic structure with graph networks, or with transformers over its limbs, is also well established:

  • for control, in NerveNet (Wang et al., 2018), Shared Modular Policies (Huang et al., 2020), Amorpheus (Kurin et al., 2021), a transformer that treats limbs as tokens, and MetaMorph (Gupta et al., 2022), a transformer over a depth-first traversal of the kinematic tree;
  • for design, in Neural Graph Evolution (Wang et al., 2019), RoboGrammar's graph value function (Zhao et al., 2020), Transform2Act (Yuan et al., 2022) and the graph-based latent space of GLSO (Hu et al., 2023);
  • for dexterous hands, in House of Dextra, which encodes each hand as a palm-rooted graph with a message-passing GNN.

The kinematic-graph branch is therefore not a contribution. The paper cites only a generic graph-pooling reference (Ying et al., 2018) for it. The "best known alternative … VAE framework" is attributed to Mitchell and Cheney (2025), which is a perspective article in genetics rather than a robot-design method.

The paper also frames its advance largely as a departure from grammar-based and modular design spaces, contrasting itself with the "manually devised parameterizations" of House of Dextra and Mirzaee et al. (2026). However, co-design research has already moved well beyond fixed grammars:

  • Voxel-based freeform design dates back over a decade, to Hiller and Lipson (2012) and the CPPN-encoded voxel robots of Cheney et al. (2013). It has since been standardised in benchmarks such as Evolution Gym.
  • Learned generative models of morphology go beyond grammar rules. Examples include the VAE latent spaces of Li et al. (2025) and the diffusion-based DiffuseBot (Wang et al., 2023).
  • Transform2Act (Yuan et al., ICLR 2022) learns a policy that both grows and controls the robot.
  • BodyGen (Lu et al., ICLR 2025) uses transformers with topology-aware self-attention for both design and control.

Against this literature, escaping a grammar is not itself a contribution. The proposed method also reintroduces one: its decoder is masked by a grammar over a hand-specified parameterisation (one root, at most five unbranched appendages, one servo per joint). The relevant comparison is therefore with these learned and freeform co-design methods, not with grammar-based hand design. I would ask the authors to position the work against this literature. The "first" claim should be restated as the first freeform, rigid, articulated manipulators evolved in a learned latent space and fabricated for dexterous tasks.

2. The evidence for the proposed representation is limited.

The central methodological claim is that the contrastive genome improves evolvability. It rests on two pieces of evidence. The first is the ablation in Fig. 7B: three evolutionary trials per condition, run for only 30 generations on the 200,000-design dataset, with standard-error bands and no statistical test. The second is the latent-space metrics in Table 11. Even so, the text describes the effect of the contrastive loss as "significant", while noting that the benefit of the hypersphere projection was not. The VAE comparison is hard to interpret. Appendix D reports that the VAE almost never recovered the number of joints and produced valid designs less than half of the time, yet no tuning of the VAE is described. The decoder is said to ensure "validity at each step". However, only 82% of decoded designs are geometrically valid (Table 11), and decoded designs then pass through up to four rejection stages (Table 14). Rejection rates during evolution are not reported. Moreover, the validity that is guaranteed derives largely from decoding into a hand-specified parametric generator. That narrows the space relative to voxel-level representations, a trade-off that deserves discussion. More trials and appropriate significance tests are needed to support the claim.

3. The framework reuses parts of House of Dextra without full acknowledgement.

The paper credits Fay et al. (2026) as related work, for the actuators, as a rotation benchmark and for the reward scheme. The released code shows considerably more reuse:

  • The rotation environment, which carries the paper's main comparison, contains the reward terms House of Dextra added to the LEAP reorientation environment. It keeps House of Dextra's position-stability threshold and weight, and its modified pose-difference weight.
  • The 16 training objects in the released code are House of Dextra's object set, with identical file names and object-type labels, yet the paper attributes them to Yin et al. (2023).
  • The simulation configuration matches House of Dextra's exactly, from timestep and friction to curriculum threshold and perturbation schedule.
  • Collision geometry is decomposed with House of Dextra's per-part CoACD thresholds, which is acknowledged only in code comments.
  • All rotation runs use a convex-hull palm close to House of Dextra's, which is not mentioned in the paper.

All three of House of Dextra's tasks reappear (in-hand rotation, tabletop grasping, and tabletop flipping that requires returning the object to its start), and the hardware stack is the same. Reusing permissively licensed code is legitimate. However, the paper presents House of Dextra as an independent baseline it outperforms while conducting its main experiment largely within House of Dextra's environment. The authors should state this reuse in the paper and the repository, and correct the attribution of shared components. For each component of the framework, they should indicate whether it was developed for this work, adapted from House of Dextra or Guo et al. (2026), or adopted unchanged. They should also fully specify how the real-world hardware and control setup were designed, including whether controller gains were tuned on hardware.

4. The comparisons with House of Dextra are not like-for-like.

The "Fay et al. (2026)" line in Fig. 7A is not a baseline run in this work. It is House of Dextra's published simulation result without fine-tuning, 1.85 rad/s, plotted on this paper's fitness axis, whereas the text compares against their fine-tuned 3.3 rad/s. The metrics also differ: net rotation over 30 s here, against continuous angular velocity over three minutes in House of Dextra. The Discussion's description of these as "otherwise equivalent state-of-the-art baselines" is therefore inaccurate. The comparison of 3.66 with 3.3 rad/s sets a fine-tuned blind student evaluated under this paper's protocol against a figure reported under another. The released fine-tuning code itself states that such scores should not be ranked against evolution fitness. The blind student's 3.66 rad/s is also well above the evolution fitness in Fig. 7A: the mean peak fitness across the five trials is about 2.3, with the upper standard-error bound near 2.6. The teacher's own fitness is not reported, and a blind student outperforming its privileged teacher by this margin needs explanation.

In the real world, the claim of being "almost twice as fast" (1.39 vs. 0.74 rad/s) compares results on two largely different sets of objects. The 1.39 rad/s is this paper's average over its own 15 test objects (16–687 g). The 0.74 rad/s comes from House of Dextra's objects, a different set of nine items of roughly 8 g to just over 100 g. Overlap appears limited to a Rubik's cube, a tennis ball and possibly a disk toy. The tennis ball is shown being rotated by both evolved rotators in Figs. 16–17 but is absent from Table 3. Neither hand was evaluated on the other's full object set, so the ratio says little about the relative capability of the two designs. Moreover, House of Dextra does not report 0.74 rad/s, although the paper and project page present it as the value House of Dextra achieved. The value appears to be derived by converting their per-object rotation times and averaging only the eight successful objects of the nine shown, out of 17 tested. Counting the failure gives approximately 0.66 rad/s. The protocols also differ: House of Dextra times a single 360∘360^\circ rotation, whereas this paper reports mean speed over two trials. Since House of Dextra's design and code are public, I would ask the authors to evaluate it within their own pipeline and on their own test objects. Failing that, they should present these figures explicitly as cross-paper numbers and explain their derivation.

5. Real-world evidence is limited.

Table 3 is based on two speed trials and three 30 s stability trials per object, and Figs. 15–19 show still frames. The project page provides a single short clip per task:

  • one rotation run on one object, a Rubik's cube;
  • a hanging run, in which the robot is initially held against the bar by hand;
  • a retrieval run.

There are no videos of the tetradactyl rotator or the picker. The only observable rotation is of a cube, resembling the block-dominated training set. According to Table 3, the tetradactyl design rotates only the cubes and one other object. It therefore cannot be ruled out that real-world performance is specific to cube-like objects. The hang and retrieve results are qualitative. The real bar is stationary, whereas it twists in simulation, and the statement that retrieval was "equally successful in reality" is not supported by trial counts. In addition, the deployed "proprioceptive" policy also receives an object-type label, which must be supplied at deployment and is not disclosed. The text also contradicts Table 3 in three places, regarding which objects the LEAP and tetradactyl policies rotated:

  • Section 3.2 states that the tetradactyl design "could not rotate the other test objects", but Table 3 shows it rotating the orange.
  • Sections 3.2 and C.1 state that the multi-object LEAP policy failed to rotate any test object, but Table 3 shows it rotating the orange.
  • Appendix C.1 states that the cube-only LEAP policy rotated "the cubes", but Table 3 shows it failing on the fabric cube.

Fig. 17 also labels two rows "Rubik's Cube 4X", and the second row shows a different object. The authors should report trial counts and success rates for every physical task and provide unedited videos for all test objects.

6. Compute, the role of the 2 million designs, and truncated training are not reported.

The 2 million designs serve only as training data for the encoder and decoder; none is evaluated on a task. With a policy trained from scratch for each design, evaluating them would be infeasible. At the released budget, it would require on the order of 170,000 GPU-hours. Evolution evaluates only on the order of 1,000–2,400 designs per campaign (population 32 over 30–75 generations), and the paper should state this distinction clearly. No compute is reported for evolution; the only compute reported is for training the encoder and decoder. In the released scripts, eight designs share a GPU and training is truncated after a fixed wall-clock time, with fitness taken from the partially trained policy. How much a design is trained therefore depends on its simulation speed and on GPU contention. This may systematically disadvantage designs with more joints and contacts, independently of their quality. The authors should report total compute, how often training was truncated, and the number of iterations reached.

7. Several method details and the multitask results do not match the code.

Appendix C.7 says the multitask normalisers are the best specialist available at the start of the trial, but in the code they are fixed constants set well below the reported specialists. Fig. 14A also appears not to equal the sum of panels B–F. Other discrepancies:

  • the fitness definition (average reward in Section 2.4, but a task metric over a separate test window in Appendix C and the code);
  • the action parameterisation (increments rather than target positions);
  • the flip direction (random in Section 3.6, fixed in Appendix C.5);
  • the dataset generator's minimum of three appendages and five joints, which contradicts Appendix B.1;
  • the number of environments and generations.

The hang policy is also retrained rather than distilled, and the pick fine-tuning departs from Table 15. With no real-robot code, retrieve code or result logs released, most headline numbers cannot be reproduced.

8. Diversity is claimed but not measured.

The abstract claims state-of-the-art "diversity", yet no diversity measure of evolved designs is reported, although the code logs one each generation. A larger design space does not establish more diverse outcomes; the winning rotator is essentially two-fingered. The authors should report such a metric or temper the claim.

9. Baselines from recent co-design work are missing.

The only external baselines are a fixed anthropomorphic hand (LEAP) and a number copied from House of Dextra, and they are used for rotation only. Pick, hang, retrieve, flip and hold have no baseline, and the multitask setting is compared only with the paper's own evolved specialists. No co-design method is run within the paper's pipeline. Claims of state-of-the-art performance and of improved evolvability therefore lack a reference point. Several recent methods address the same problem and should be benchmarked on at least the rotation task, and ideally on all tasks, under a matched evaluation budget:

  • House of Dextra (Fay et al., 2026), whose code and generator are public, as the direct dexterous-hand co-design baseline.
  • Li et al. (2025), a VAE latent space with CMA-ES and a universal controller, which tests whether the contrastive genome and per-design PPO improve on the authors' own prior approach.
  • Learned co-design methods that move beyond grammars: Transform2Act (Yuan et al., 2022) and the transformer-based BodyGen (Lu et al., 2025).
  • Latent-space manipulator co-design in the manner of Ikemura et al. (2026).
  • Simple controls that isolate the value of the learned latent space: CMA-ES and random search applied directly to the design parameters.

Without these comparisons, the gains in Fig. 7 cannot be attributed to the proposed representation rather than to the search budget or the evaluation protocol.

Questions for authors
  1. The 0.74 rad/s figure comes from House of Dextra's own object set, not the 15 objects used here. Can you evaluate House of Dextra's optimised hand within your own pipeline and on your own test objects, and explain how the 0.74 rad/s figure was obtained?
  2. Which components of the framework (environments, objects, simulation settings, collision modelling, palm generation, hardware and control) were adopted from House of Dextra or Guo et al. (2026), and which were developed for this work?
  3. How does the work relate to Evolution Gym, DERL, DiffuseBot, Pan et al. (2020), RoboGrammar and N-LIMB, and what is the precise sense in which this is the first de novo manipulator design?
  4. Can you report more evolutionary trials with significance tests for the ablations, and describe how the VAE baseline was tuned?
  5. What is the total compute for evolution, how often was per-design training truncated, and how are designs that simulate slowly treated?
  6. Can you provide trial counts, success rates and unedited videos for all real-world tasks and test objects, and disclose the object-type input used at deployment?
  7. Can you report a diversity measure for evolved populations and selected designs?
  8. Why are confidence intervals and baselines reported only for rotation? Pick, hang, retrieve, flip, hold, and multitask each appear to rest on a single evolutionary run, with no comparison to any baseline. How are gains validated over run-to-run noise?

Specifically: (a) Can you report 95% confidence intervals over multiple independent runs for every task, for both final fitness and success rate? (b) Can you give real-world success rates with confidence intervals (e.g. binomial), stating the number of attempts? (c) How does your method compare with established co-design approaches on the same tasks, such as BodyGen (Lu et al., ICLR 2025), Transform2Act (Yuan et al., ICLR 2022), a shared transformer controller like MetaMorph (Gupta et al., ICLR 2022) or Li et al. (2025), and House of Dextra's graph heuristic search re-run in your pipeline? (d) Does the learned latent space actually help? How does it compare with random search applied directly to the raw design parameters (39 + 12 per segment) under the same evaluation budget?

4 comments

The reviewer has accused us of plagiarism. So, we will begin by refuting this erroneous accusation.

The reviewer is concerned that we have replicated the “House of Dextra” paper (Fay et al., 2026), one of the baselines we compared our approach against. To ensure a fair comparison, we have of course attempted to reproduce the experiment from Fay et al. (2026) to the best of our ability. In-hand rotation is a standard manipulation benchmark. The set of training objects already exists in Yin et al. (2023), the reward function follows Handa et al. (2023) with two modifications from Fay et al. (2026), a penalty on object displacement beyond 3 cm and a smaller pose-difference weight (0.1 instead of 0.3), and the basic hardware already exists in Shaw et al. (2023). We also could not successfully train using the exact reward function supplied in their released code directly: their released rotation environment turns on five contact and grasp shaping terms unrelated to in-hand rotation (contact, multi-finger grasp, approach, fingertip distance and finger collision), so we turned these terms off. Moreover, unlike prior work, we do not assume standardized body modules and so do not constrain the position and orientation of actuators or the geometry of the segments between them. With this careful experimental design, we evolved robots that are more complex, diverse and dexterous than those generated by otherwise equivalent state-of-the-art baselines which operate within a more restrictive design space (Fay et al., 2026) or assume an anthropomorphic layout (Shaw et al., 2023).

The robots that evolved in our paper are themselves novel. That is, they have never existed before. Neither their kinematic trees nor geometries have ever been reported in the literature. Indeed, they could not have arisen in Fay et al. (2026) because, as we explained in our manuscript, their design space was limited to inward-bending stacks of cubic modules. The shape and articulation of each finger was largely presupposed, and their position was locked in plane about a 2D palm. As a result, a limited diversity of designs could arise, and differences between unoptimized and optimized morphologies could be difficult to discern. In other words, evolution could not radically change the body plan, new “species” could not emerge.

However, we would like to emphasize our admiration and respect for Fay and her colleagues. The robots that they evolved could manipulate objects, which was remarkable and inspiring. We cite them rigorously throughout our paper.

We agree with the reviewer that there are many papers that have evolved virtual creatures. Most evolved locomotors, some evolved gross manipulators, few evolved dexterous manipulators. Often these environments defy the laws of physics and common sense. Our paper is about evolving real robots that solve real problems in the real world. With respect, we believe that we have appropriately situated our work in the literature. We kindly refer the reviewer to our Introduction.

We respectfully disagree with the reviewer that our evidence in the real world is limited. Our experiments in the real world were much more comprehensive than all prior work in evolutionary robotics, perhaps with the exception of Yu et al. (2026), who evolved agile locomotors not dexterous manipulators.

We do however agree with the reviewer that a baseline directly evolving robots in token space was missing from our original submission. We performed this additional experiment and report the results in our revised manuscript (in Fig. 7A). We found that the best designs that evolved in token space were significantly worse than those that evolved in latent space.

Our diversity claim is based on the number of possible kinematic trees and geometries that can and did evolve in our experiments compared to those that were possible in prior work. Geometry and joint angles have a huge effect on dexterity but were presupposed and held fixed in prior work. Freely evolving these morphological features adds enormous diversity to design space. But, even if we ignore this and compare only the number of topologies that could be generated in prior work, Fay et al. (2026) could realize only 46 configurations (3-5 digits with 3-5 joints each). In our work there are 2,601 possible configurations (1-5 digits with 25 joints total). Again, however, we emphasize that freeform geometries and joint angles increases diversity exponentially beyond what could be achieved with the modular hardware used in prior work.

Finally, Fay et al. (2026) did not fully identify their test objects, and purchase links were not provided. This made it difficult to obtain the exact same items. However, while it was not possible to make an exact comparison, we could and did make a conservative comparison by considering a more diverse set of test objects with a broader range of masses, materials, and shapes (Fig. 21).

We would also like to respectfully clarify several factual errors and mischaracterizations in the review. Below, we address each point with reference to the methods, results, and claims reported in our paper.

  1. “To my knowledge, using a contrastive objective to build a searchable latent genome for robot morphology is new. This allows a VAE to map similar phenotypes to distant genotypes.”

We thank the reviewer for recognizing one of our main contributions. However, the next sentence describes the opposite of our finding. Contrastive learning maps similar phenotypes to nearby genotypes, improving evolvability. The VAE can not preserve this relationship. Our proposed representation is not a VAE, the VAE is a baseline.

2, “House of Dextra (Fay et al., 2026) co-designed, fabricated and deployed rigid, servo-driven hands for in-hand rotation, grasping and flipping with the same XL330 actuators”

This statement overstates HoD’s real-world results. HoD demonstrated in-hand rotation only on physical robots; its grasping and flipping results were simulation-only. Our work fabricated five designs and deployed them zero-shot across four tasks: rotation, picking, hanging, and retrieval.

3.“The kinematic-graph branch is therefore not a contribution.”

This is misleading. We never claim the kinematic-graph encoder as a contribution in our paper. Our encoder also does not rely solely on the kinematic graph. It combines a GNN encoding of that graph with a 3D CNN encoding of the voxel geometry. One of our main contributions is using contrastive learning to build a searchable latent representation for morphological evolution.

4.“The "Fay et al. (2026)" line in Fig. 7A is not a baseline run in this work. It is House of Dextra's published simulation result without fine-tuning, 1.85 rad/s, plotted on this paper's fitness axis, whereas the text compares against their fine-tuned 3.3 rad/s.”

Fig. 7 reports results before finetuning, while Section 3.2 discusses results after finetuning. Using HoD’s corresponding number at each stage is internally consistent.

5.“The teacher's own fitness is not reported, and a blind student outperforming its privileged teacher by this margin needs explanation.“

Fig. 7A reports mean peak fitness across evolutionary trials before distillation and finetuning. The 3.66 rad/s result is measured after the student received additional PPO finetuning. These two numbers therefore do not establish the claimed student–teacher performance gap.

6.“Compute, the role of the 2 million designs, and truncated training are not reported”

This misrepresents the role of the 2M designs. Section 2.2 explicitly states that this synthetic dataset was used to train the encoder and decoder, and Appendix B.1 describes its generation. We never claim that all 2M designs were evaluated on manipulation tasks.

7.“the authors figure on the website reusing the same 4 House of Dextra hands for in hand rotation without accreditation”

This claim is false. We did not use images of HoD’s four hands on our website. Please identify the specific image or website commit that supports this allegation.

8.“The authors pass these hardware designs as their own, but the hardware is the exact same but with voxels applied to randomize the 3d printed parts.”

The claim is not true. We use off-the-shelf Dynamixel actuators because they are compact and affordable. Dynamixel actuators are also used in HoD (Fay et al., 2026) and LEAP Hand (Shaw et al., 2023). We do not claim the motors as our contribution. Our contribution is a framework for evolving freeform manipulators and automatically converting them into buildable robots. We independently developed our freeform design space around these actuators and the requirements of manipulation tasks.

I thank the authors for their response. While a token-space baseline was added, the majority of the questions and experiments requested in my review were not addressed:

  • Repeated evolutionary runs with confidence intervals and significance tests. Pick, hang, retrieve, flip, hold and multitask rest on a single run each, the ablations in Fig. 7B on three runs, and trial counts for the full-scale curves in Fig. 7A are not stated.
  • Comparisons with established co-design methods (e.g. BodyGen, Transform2Act, Li et al. 2025, or House of Dextra's search) run within the authors' pipeline.
  • An evaluation of House of Dextra's public design on the same test objects as the evolved designs.
  • Trial counts and success rates for the physical hang and retrieve tasks.
  • Total evolution compute in the paper, and how often per-design training was truncated.
  • A diversity measure over evolved designs; Sect. 3.10 counts topologies over all evaluated designs (720 of 12,960).
  • Disclosure of the object-type input used at deployment.

The novelty and contribution also remain limited. Latent-space morphology search with CMA-ES, fabrication-ready blueprints, and zero-shot deployment of fabricated manipulators have direct precedent in Li et al. (2025), Guo et al. (2026) and House of Dextra. Per-design PPO with teacher–student distillation, grammar-constrained decoding and kinematic-graph encoding are also well established. The contribution is therefore incremental. Strong concerns for proper attribution also remain and can be addressed in the ethics review.

Reviewer_CLocOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies joint optimization of robot morphology and control within a parameterized hub-and-appendage design space. It integrates a contrastively learned morphology embedding, an autoregressive decoder, CMA-ES, and PPO with automatic conversion to manufacturable designs. Experiments cover six task types, with five fabricated robots tested on four. A shared morphology is also evaluated across multiple tasks using separate task-specific policies.

Critical strengths
  1. The expanded morphology space and physical validation make this a useful advance in robot co-design. The didactyl robot rotates all 15 unseen physical test objects, and the picker succeeds in 80% of 240 attempts (Tables 3 and 6). These results show that the generated designs support useful manipulation after fabrication.

  2. The reconstruction-only ablation supports the value of contrastive training. It has better joint-count reconstruction and slightly better geometric validity than the matched proposed model, yet performs worse in evolutionary search (Table 11; Fig. 7B). This provides evidence that the learned neighborhood structure matters beyond reconstruction accuracy.

Critical weaknesses
  1. The VAE comparison is not conclusive. The baseline suffers posterior collapse and reaches only 6% joint-count accuracy (Sec. 3.9; Table 11). Poor search with this model does not establish an advantage over a functioning VAE. VAE-specific regularization settings and attempts to prevent collapse are not reported.

  2. The headline real-world comparison is not matched. The 1.39 versus 0.74 rad/s comparison uses different object sets and evaluation protocols from Fay et al. (2026); Table 3 does not evaluate their design under the same test. This weakens the claimed near-twofold improvement. Baseline reporting also needs correction: Sec. 3.2 says the LEAP policy trained on multiple objects fails on every test object, but Table 3 reports successful rotation of the orange. arxiv.org

Questions for authors
  1. What KL weight and training schedule were used for the VAE? Does the search advantage remain when posterior collapse is avoided? This requests clarification of the existing baseline and evidence supporting the representation claim.

  2. How was the 0.74 rad/s figure for Fay et al. computed? Can the comparison be repeated on the same objects with matched controller-training budgets and evaluation protocols? This is additional evidence needed to substantiate the performance claim.

  3. How should the claim of no assumed body structure be reconciled with Appendix A’s fixed central hub and up to five nonbranching appendages? Please state these structural assumptions when defining the contribution.

3 comments

We thank the reviewer for their time and careful reading of our paper. We appreciate that the reviewer found our work to be a useful advance.

Below we address the stated weaknesses and questions, one by one, also referencing how our manuscript was modified in response. The updated manuscript has been uploaded to OpenReview with additions marked in red text. We believe all changes have significantly improved the paper, and we thank the reviewer for prompting them.

 

W1. The VAE comparison is not conclusive. The baseline suffers posterior collapse and reaches only 6% joint-count accuracy (Sec. 3.9; Table 11). Poor search with this model does not establish an advantage over a functioning VAE. VAE-specific regularization settings and attempts to prevent collapse are not reported.

In Results, we added the following (new text italicized):

“Despite our best attempts (detailed in Appx. D.3), the VAE suffered posterior collapse...”

And in Appx. D.3, we added description of the VAE training settings and our attempts to prevent posterior collapse:

“We tested several VAE configurations (Table 13) to address poor reconstruction. Each VAE variant was trained for 50 epochs with KL annealing (Bowman et al., 2016) and free-bits (Kingma et al., 2016). The KL weight β increased linearly from 0 to 0.01 over the first 10 epochs and then remained fixed. We trained on both 200k and 2M designs with a free-bits threshold of ϵ = 0.03 nats per dimension, and also tested a higher threshold of ϵ = 0.05 on the 2M dataset.

“As shown in Table 13, increasing the dataset size from 200k to 2M improved geometric validity from 0.33 to 0.99, but Joint acc. remained at 0.06. Raising the free-bits threshold to ϵ = 0.05 improved Joint acc., but it remained low at 0.14. To further investigate the collapse, we compared the true and decoded joint counts on the test set of 2,000 designs, as shown in Fig. 24. The VAE (2M) frequently decoded inputs with different joint counts into designs with around 24 joints, while our model closely reproduced the true counts. Thus, increasing the dataset size and applying KL annealing and free bits did not resolve the collapse in the tested VAE configurations. These results suggest that these configurations are poorly suited to learning a representation that supports diverse morphologies and effective latent-space search.”

References:
Bowman et al. (2016) CoNLL. https://aclanthology.org/K16-1002/
Kingma et al. (2016) NeurIPS. https://arxiv.org/abs/1606.04934

 

W2. The headline real-world comparison is not matched. The 1.39 versus 0.74 rad/s comparison uses different object sets and evaluation protocols from Fay et al. (2026); Table 3 does not evaluate their design under the same test. This weakens the claimed near-twofold improvement. Baseline reporting also needs correction: Sec. 3.2 says the LEAP policy trained on multiple objects fails on every test object, but Table 3 reports successful rotation of the orange. arxiv.org

We agree with the reviewer’s assessment. For context, Fay et al. (2026) did not fully identify their test objects, and purchase links were not provided. This made it difficult to obtain the exact same items. However, while it was not possible to make an exact comparison, we could and did make a conservative comparison by considering a more diverse set of test objects with a broader range of masses, materials, and shapes (Fig. 21). We have revised our manuscript to clarify this and the limited success of the LEAP policy as follows.

In Sect. 3.2, we now say (new text italicized):

“...almost twice as fast as the best design reported by Fay et al. (2026), which achieved 0.74 rad/s on average. However, it is important to note that this is not a perfect comparison because the unseen test objects used by Fay et al. (2026) were not precisely specified beyond that they weighed from 8 to over 100 g. The evolved tetradactyl robot rotated cubes at 1.78 rad/s and the orange 0.9 rad/s but failed to rotate the other test objects. After the same training process, the anthropomorphic hand (Shaw et al., 2023) failed to rotate any of the previously unseen test objects except for the orange, which was rotated at 0.2 rad/s.”

In Appx. C.1, we added the following new text:

“These objects were deliberately selected to assess generalization across different weights, materials, and shapes. They included common household items, artificial fruits, and a small number of objects from the YCB dataset (Calli et al., 2015), with weights ranging from 16.1 to 687.3 g.”

References:
Calli et al. (2015) IEEE Robotics & Automation Magazine.

Q1. What KL weight and training schedule were used for the VAE? Does the search advantage remain when posterior collapse is avoided? This requests clarification of the existing baseline and evidence supporting the representation claim.

Please see our response to W1. In Appx. D.3, we now report that VAEs are trained with a 10-epoch KL annealing schedule increasing β\beta from 0 to 0.01, and 50 training epochs. When training the VAE (200k) that we reported in our original submission, we implemented the free bits method together with the KL annealing method to try to avoid posterior collapse. In our revised manuscript, we added additional experiments that varied the dataset size and free-bits threshold and implemented the contrastive VAE suggested by another reviewer. Despite our best efforts, the VAE continued to suffer posterior collapse.

 

Q2. How was the 0.74 rad/s figure for Fay et al. computed? Can the comparison be repeated on the same objects with matched controller-training budgets and evaluation protocols? This is additional evidence needed to substantiate the performance claim.

Please see our response to W2. A perfect comparison is not possible because the objects used by Fay et al. (2026) were not specified. Without this information, we attempted to the best of our abilities to make a conservative comparison.

Regarding the 0.74 rad/s figure for Fay et al., we have added the following explanation to Appx. C.1:

“In our experiments, we used OptiTrack to record object orientation and calculated average angular velocity over the measurement period. In contrast, Fay et al. (2026) reported manually measured times for one full rotation. To express their results in the same units, we converted each reported time using ωi=2π/Ti\omega_i = 2\pi/T_i and averaged the resulting angular velocities across their reported objects to obtain 0.74 rad/s.”

 

Q3. How should the claim of no assumed body structure be reconciled with Appendix A’s fixed central hub and up to five nonbranching appendages? Please state these structural assumptions when defining the contribution.

We claim that “Unlike prior work however, we do not assume fingers about a palm and do not constrain the position and orientation of actuators or the geometry of the segments between them.” We believe this is supported by Fig. 3 and we have added Fig. 11 to the Appendix to further illustrate the geometric and kinematic freedom available in design space.

To reiterate the structural assumptions and how they accommodate de novo manipulator design, we have added the following new text to Discussion:

“Finally, evolution was constrained to morphologies with no more than five appendages and at most 25 joints. Inside these convenient bounds, however, an astonishing variety of strange and familiar forms grew out of the genotypes that were expressed during evolution. Fig. 3 shows a coarse-grained picture of one part of the genetic hypersphere, interpolating between four evolved genotypes. Some of the decoded designs resemble an elephant’s trunk, others resemble a bird’s foot or beak, yet others resemble a primate’s hand. However, many do not have a natural analogue.”

Table 13: VAE variants of the latent genome.

ModelApp. acc.Joint acc.ValidRecall@1Edit ρTopology 10-NN acc.Length 10-NN rVolume 10-NN r
ValiditySmoothness
VAE (200k)0.440.060.330.010.93 ± 0.130.020.790.79
VAE (2M)0.440.060.990.000.98 ± 0.070.080.600.54
VAE (2M, ϵ\epsilon = 0.05)0.440.140.950.020.98 ± 0.080.080.780.72
VAE + Lcon\mathcal{L}_{\mathrm{con}} (2M)0.440.010.890.920.97 ± 0.100.000.960.88
Reviewer_voTsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an end-to-end morphology-control co-design framework for evolving robotic manipulators and deploying them zero-shot in the real world. The core contributions include a representation learning pipeline that embeds a parametric morohospace onto a hypersphere using contrastive learning and decodes latent genotypes via a grammar-masked autoregressive Transformer. The second one is an automated CADify, privilege to proprioception policy distillation, and hardware fabrication pipeline. The approach is validated across 6 simulated manipulation tasks and a multitask setting.

Critical strengths
  1. This work bridges freeform geometry evolution, automated motor embedding and self-collision carving, policy distillation, and physical fabrication. Five distinct physical manipulators are built and achieve continuous zero-shot in-hand rotation across 15 diverse unseen objects. The result supports the claim that evolved non-anthropomorphic morphologies can reliably execute contact-rich manipulation in reality.
  2. This paper demonstrated that high token reconstruction accuracy alone does not guarantee effective black-box search under CMA-ES. Enforcing pose invariance and local topological neighbourhood coherence via contrastive learning is shown to be critical for fuiding evolutionary optimisation.
Critical weaknesses

It appears that the paper's own morphospace is strictly bounded by a hand-crafted template, as shown in Appendix A. Because any grammar-compliant parameter vector or token sequence can be directly compiled into a robot, the central representation learning claim could be better supported by adding some direct parameter or token-space optimisation baseline (like a genetic algorithm or even a random search). Furthermore, the VAE baseline in Table 11 seems to suffer from severe posterior collapse. It's not clear whether some useful configurations (e.g., KL annealing, beta-tuning, etc.) have been applied to make VAE "the best known alternative".

Questions for authors
  1. How does evolving in the learned latent space compare against directly optimising the underlying continuous design parameters or discrete design tokens under the same evaluation budget?
  2. Did you tune the KL weight \beta or apply any techniques to prevent posterior collapse in the VAE baseline?
  3. If all ablation variants in Fig. 7B use a unified initialisation protocol and 2M vs. 200k models are compared at Gen 30, do the differences remain statistically significant?
3 comments

We thank the reviewer for their close examination of our work and helpful feedback. Below we address the stated weaknesses and questions, one by one, also referencing how our manuscript was modified in response. The updated manuscript has been uploaded to OpenReview with additions marked in red text. We believe all changes have significantly improved the paper, and we thank the reviewer for prompting them.

 

W1. It appears that the paper's own morphospace is strictly bounded by a hand-crafted template, as shown in Appendix A. Because any grammar-compliant parameter vector or token sequence can be directly compiled into a robot, the central representation learning claim could be better supported by adding some direct parameter or token-space optimisation baseline (like a genetic algorithm or even a random search).

We have followed the reviewer’s suggestion and conducted additional evolutionary trials directly in token space. We used grammatical evolution (O’Neill & Ryan, 2001), which is to the best of our knowledge the most appropriate choice. The results can be seen in Fig. 7A.

In Results, we added a description of this new experiment:

“Finally, we completely removed the latent genome and ran evolution directly in token space. To do so, we employed grammatical evolution (O’Neil & Ryan, 2001), ensuring that every evolved token sequence produced a valid design. The best designs that evolved in token space were significantly worse than those that evolved in latent space (Fig. 7A).”

References: O’Neill & Ryan (2001). IEEE Trans. on Evo. Comp. https://ieeexplore.ieee.org/abstract/document/942529

 

W2. Furthermore, the VAE baseline in Table 11 seems to suffer from severe posterior collapse. It's not clear whether some useful configurations (e.g., KL annealing, beta-tuning, etc.) have been applied to make VAE "the best known alternative".

We have expanded Appx. D.3 to describe the VAE training settings and our attempts to prevent posterior collapse using KL annealing and free bits. We have also added additional experiments to test VAE performance with more training data and a higher free-bits threshold to further limit posterior collapse. Please see Table 13 below.

In Results, we added the following (new text italicized):

“Despite our best attempts (detailed in Appx. D.3), the VAE suffered posterior collapse…”

And in Appx. D.3, we added the following new text:

“We tested several VAE configurations (Table 13) to address poor reconstruction. Each VAE variant was trained for 50 epochs with KL annealing (Bowman et al., 2016) and free-bits (Kingma et al., 2016). The KL weight β increased linearly from 0 to 0.01 over the first 10 epochs and then remained fixed. We trained on both 200k and 2M designs with a free-bits threshold of ϵ = 0.03 nats per dimension, and also tested a higher threshold of ϵ = 0.05 on the 2M dataset."

“As shown in Table 13, increasing the dataset size from 200k to 2M improved geometric validity from 0.33 to 0.99, but Joint acc. remained at 0.06. Raising the free-bits threshold to ϵ = 0.05 improved Joint acc., but it remained low at 0.14. To further investigate the collapse, we compared the true and decoded joint counts on the test set of 2,000 designs, as shown in Fig. 24. The VAE (2M) frequently decoded inputs with different joint counts into designs with around 24 joints, while our model closely reproduced the true counts. Thus, increasing the dataset size and applying KL annealing and free bits did not resolve the collapse in the tested VAE configurations. These results suggest that these configurations are poorly suited to learning a representation that supports diverse morphologies and effective latent-space search.”

 

Table 13: VAE variants of the latent genome.

ModelApp. acc.Joint acc.ValidRecall@1Edit ρTopology 10-NN acc.Length 10-NN rVolume 10-NN r
ValiditySmoothness
VAE (200k)0.440.060.330.010.93 ± 0.130.020.790.79
VAE (2M)0.440.060.990.000.98 ± 0.070.080.600.54
VAE (2M, ϵ\epsilon = 0.05)0.440.140.950.020.98 ± 0.080.080.780.72
VAE + Lcon\mathcal{L}_{\mathrm{con}} (2M)0.440.010.890.920.97 ± 0.100.000.960.88

References:
Bowman et al. (2016) CoNLL. https://aclanthology.org/K16-1002/
Kingma et al. (2016) NeurIPS. https://arxiv.org/abs/1606.04934

Q1. How does evolving in the learned latent space compare against directly optimising the underlying continuous design parameters or discrete design tokens under the same evaluation budget?

Please see our response to W1. Following the reviewer’s suggestion, we evolved robots directly in token space under the same evaluation budget. Evolution in token space was significantly worse than evolution in latent space. Thank you for suggesting this informative baseline.

 

Q2. Did you tune the KL weight \beta or apply any techniques to prevent posterior collapse in the VAE baseline?

Yes, please see our response to W2. We used KL annealing and free bits when training the VAE (200k) in our original submission. We added additional experiments that varied the dataset size and free-bits threshold and implemented the contrastive VAE suggested by another reviewer. However the posterior collapse persisted.

 

Q3. If all ablation variants in Fig. 7B use a unified initialisation protocol and 2M vs. 200k models are compared at Gen 30, do the differences remain statistically significant?

During the rebuttal period thus far we have retrained all models with the larger dataset consisting of 2M example body plans. We are currently rerunning all five independent evolutionary trials with each model. We will then completely remove the 200k models from the paper. Because each evolutionary trial requires approximately 33 hours on 4x H100 GPUs, these experiments are still in progress. We will update this response with the results and statistical analysis as soon as they are available, before the end of the rebuttal period.

Thank you for the effort put into the revisions. My initial concerns have been well addressed in this case.

Having also read through the broader discussion with the other reviewers, I see the authors' edits to clarify the evaluation protocol differences relative to Fay et al. (2026). While the real-world comparison against prior work remains somewhat indirect due to differing test object sets, the end-to-end physical demonstrations across multiple tasks and the added token-space baseline make sense. I will maintain my score at the moment.

Comments on the paper

2 comments

Dear reviewers,

As we only have a few more days to go before the discussion window closes, please go through the author responses and acknowledge them. You may highlight aspects of your initial review comments that have been satisfactorily addressed by the responses, as well as those which require further clarification.

Thanks,

AC

AuthorsOpenReview ↗

Summary of Revisions

Summary of Revisions

We thank all reviewers for their careful reading and constructive feedback. We appreciate their recognition of the value of learning a searchable representation of robot morphology and validating the evolved designs on physical robots. The discussion helped us tighten our claims, expand the experimental comparisons, and clarify the scope of our conclusions. We summarize the main revisions below, with detailed responses in the individual reviewer threads.

1. Scope of the contributions and evaluation protocols (CLoc, fSqK, Yumh)

We tightened our claims to describe the paper’s contributions more precisely, emphasizing the searchable morphology representation, the flexibility of the design space, and the zero-shot deployment of evolved robots across four physical tasks. We also clarified how our simulation rotation metric differs from that of House of Dextra and how the physical evaluations differ in test objects and measurement protocols (Introduction, Sect. 3.2, Appx. C.1).

2. Additional VAE experiments (voTs, CLoc, fSqK)

We expanded the VAE evaluation to include additional training set sizes and free-bits thresholds, as well as a contrastive VAE. We documented the training settings and reported how these configurations affected latent retrieval and morphology reconstruction (Appx. D.3, Table 13, Fig. 24).

3. Multitask comparisons with specialists and LEAP Hand (fSqK)

We added a task-by-task comparison of the evolved multitask morphology, evolved specialists, and LEAP Hand with both task-specific policies and a distilled multitask policy. These results clarify the strengths and limitations of the evolved multitask morphology across the five tasks (Appx. C.7, Table 11).

4. Topological diversity observed during evolution (fSqK)

We added a quantitative analysis of the distinct topologies encountered during the rotation searches and across the other task environments (Sect. 3.10).

5. Factual clarifications regarding Reviewer Yumh’s allegations

Reviewer Yumh's allegations that we reused images of the House of Dextra robots on our project website, reused their codebase, or reproduced their hardware designs by applying random voxels are false. Our robots' morphologies were evolved independently in our own design space. Our point-by-point response to Reviewer Yumh addresses these allegations and documents the other factual errors we identified in the review.

We thank all reviewers again for their careful reading, constructive feedback, and engagement throughout the discussion. Their suggestions helped us improve the paper through additional experiments, more comprehensive comparisons, and clearer presentation. We have also shared our code repository and project website, along with the commit histories.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.