- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In this paper, the authors introduce Co-Design Gym, which extends Gymnasium with an explicit embodiment space and design arguments, and introduces native, extended, and reframed co-design environments across 20 families and over 85 presets. Besides, four adapted optimizers are introduced and compared under a nominal four-hour budget. In my opinion, the contribution is principally hybrid benchmark infrastructure (which covers various robotic problems and some application problems), rather than a new optimization algorithm or substantive co-design theory.
Critical strengths
-
This benchmark provides a potentially useful common interface for co-design, which includes explicit design spaces, batched validation, and separation of training-time and evaluation-time embodiment changes address practical obstacles to comparing co-design methods. This is a real infrastructure contribution beyond domain-specific platforms such as EvoGym and SoftZoo.
-
This benchmark includes contact-rich manipulation and mixed discrete/continuous applications broaden evaluation beyond locomotion. Five-seed reporting is useful and convincing. It is more applicable than existing benchmarks.
Critical weaknesses
-
The API and task taxonomy are understandable. However, the evaluated-preset count differs between Figure 8 (16) and Section 6 (18), Figure 28 mentions unfinished runs, and Appendix D retains “temperature of X”; Table 3 omits the LLM rows described in its caption. I don't think the author took the experiment seriously.
-
The benchmark does not demonstrate the quantity it claims to measure: benefit from embodiment-policy co-optimization. Tables 1–2 compare joint optimizers but omit matched control-only training on the base embodiment, random-design controls, and design-only evaluation with a controlled controller. Consequently, poor task performance and changing method rankings cannot distinguish difficult co-design from difficult policy learning or infeasible design sampling. Appendix A provides task descriptions rather than complete bounds, constraints, rewards, and instance splits. The soft-body and shape-changing environments are prominent contributions but are absent from the main scoreboard. The elementary supremum statements in §3/Appendix F do not fill this empirical gap. A smaller, fully specified core suite with measured design gains would be more convincing than an unvalidated headline preset count.
I believe the true value of a co-design benchmark should be to provide a reliable standard for measuring exactly how much benefit is derived from the coordinated optimization of morphology and strategy, rather than simply accumulating the number and variety of tasks.
-
Baseline modifications and evaluation choices undermine the claimed representative comparison.** LOKI loses its robot-specific tokenization and uses six rather than forty clusters; the original LOKI relies on a learned morphology space and clustered policy reuse. FastTD3 rejects every NeroGraspAll design, while PPO+NGOpt scores designs using a changing controller. There is no demonstrated validation of these adaptations or adequate disclosure of comparable model sizes, tuning, interactions, and actual preprocessing costs. Four-hour wall time is a useful engineering metric, but cannot alone establish an algorithmic ranking. FastTD3 also generates an embodiment per evaluation rollout, whereas other baselines evaluate selected designs: clarify whether all methods solve the same fixed-design or observation-conditioned design problem. These issues affect the central scoreboard, not merely presentation.
-
The authors’ approach to modeling co-design problems is limited and unclear (whether it involves a two-layer cyclic optimization approach like Evogym or a single-layer joint optimization approach like Transform2Act), and they do not provide a detailed description of the design spaces and characterization methods for morphological and control parameters. This makes it quite difficult for users to adapt the method using their own characterizations. I encourage the authors to focus on robot co-design.
Questions for authors
For main concerns, please refer to the Critical Weaknesses section. Here are some minor questions:
-
For Multi-Robot Truck Unloading, I don't think this approach to modeling integrated control strategies fully leverages the advantages of multi-agent co-design.
-
For different type of robots, is there a unified approach to modeling design spaces—one that goes beyond simply vectorization and fixed topology settings?
6 comments
Response to Reviewer ecMz (2/3)
W3: Baseline adaptations and evaluation choices
We appreciate the close look at our baselines; each adaptation is discussed below, with new evidence where it was needed. All of our baseline algorithms were run under a matched budget to reflect practical computational choices.
LOKI. LOKI's clustering approach with 40 clusters and controllers in parallel made it infeasible to apply within the budget provided for the other baseline algorithms, so we reduced the number of clusters to 6, so that each individual controller receives sufficient training time. Its robot-specific tokenization is designed for the limb-based robots on which it was engineered (one token per limb [13]) and does not transfer to designs such as meshes, layouts or Pokémon teams, which further motivates the need for a common API and representation. To test the validity of this reduction, we ran LOKI with its full 40-cluster specification on two presets: Microgrid-Random, where 6-cluster LOKI achieves the highest percentage of the achievable reward, and SolarCleaner-Reach, where it achieves the lowest (App. C.5, Table 13):
| Preset | 6 clusters (norm. %) | 40 clusters (norm. %) | Raw return, 6 → 40 clusters |
|---|---|---|---|
| Microgrid-Random (LOKI's best) | 52.1 ± 0.2 | 51.8 ± 0.4 | −25,107 → −25,262 |
| SolarCleaner-Reach (LOKI's worst) | 0.07 ± 0.02 | 0.17 ± 0.01 | 0.0061 → 0.0155 |
With 40 clusters, LOKI's standing is unchanged on both presets - Microgrid-Random is unchanged and SolarCleaner-Reach improves marginally, even though each of the 40 controllers receives roughly 7× fewer PPO updates within the same budget.
FastTD3 on NeroGraspAll. FastTD3 alone never learns to search the space: 99.8% of the designs it proposed during training, and all of those at evaluation, failed structural validation, compared with 0.7% for CMA-ES and 10% for PPO+NGOpt (Table 12). We report this as a finding about RL-only co-design on constrained design spaces.
PPO+NGOpt scoring designs with a changing controller. This indeed is a defining trait of alternating co-design: the PPO policy is conditioned on the design, so a single controller learns across all the designs NGOpt proposes, and since it keeps improving while designs are proposed, each design is scored by how well the current controller uses it. This lets the design search and the design-conditioned policy adapt to each other over time. We agree it is worth contrasting with scoring designs under a fixed controller, and the new design-only control (App. C.2) does that.
Model sizes, interactions and wall-clock budget. We agree that wall-clock time alone cannot establish an algorithmic ranking. We discussed the choice of budgeting metric at length internally; to our knowledge, there is no established standard for comparing algorithms as different as those in our study, and every candidate metric introduces its own bias (environment timesteps favor sample-efficient methods; FLOPs do not account for parallelization). We ultimately chose wall-clock time because it most closely reflects practical constraints, and we now make all relevant cost measures visible: trainable parameters and environment steps per method (Table 10), per-preset steps and sample efficiency (Table 11), and design-rejection rates and stability (Table 12). Compared at matched environment steps instead of matched time, the leader changes on 22 of 35 presets, and no method dominates in either view (App. C.4).
Fixed-design vs observation-conditioned design. The common denominator is that each method explores the same design space and is given the same wall-clock budget; we deliberately do not restrict other aspects of how the methods explore the design and policy space, as we aim to show approaches that employ a variety of strategies. To show how to use RL for co-design with minimal method modifications, we use FastTD3, which samples a new design at the start of each episode and conditions the subsequent steps on that design; the code change is minimal compared to policy search, so it indicates how well RL can handle co-design on its own. At evaluation, its deterministic policy chooses one design per evaluation world from that world's initial observation, and the design then stays fixed for the episode. PPO+NGOpt instead shows how RL can benefit from combination with global optimization and population-based sampling: NGOpt samples a design and passes it to PPO, which conditions the policy on the selected design but does not need to learn design-sampling choices itself. Hence off-policy FastTD3 reuses scarce design decisions through replay in a single-level optimization, while on-policy PPO gives NGOpt up-to-date design fitness in a two-level formulation - the former favoring formulation simplicity, the latter following the most commonly used two-level co-design setup. CMA-ES further simplifies the single-level setting, while LOKI tests a more complex single-level setup than RL-only FastTD3.
Response to Reviewer ecMz (3/3)
W4 / Q2: Modelling approach and design-space representations
Our goal is to accommodate both styles, so that method developers can choose the organization that suits their approach best. Section 3 treats co-design as a joint problem over embodiment and policy, which nested methods also fit. The API supports both two-level methods, in which an outer design optimizer proposes embodiments and an inner learner trains a controller for them, as in EvoGym [14], and single-level methods, in which one agent or optimizer chooses both the embodiment and the control, as in Transform2Act [15]. The baselines illustrate this range: PPO+NGOpt alternates an outer design optimizer with PPO training (two-level); FastTD3 treats embodiment parameters as actions at the start of each rollout (single-level); and CMA-ES searches over design and controller parameters as one vector (single-level). All baselines run on the same environments under the same evaluation protocol, making it possible to compare the styles directly without mandating a single way to organize the optimization loop(s). We now state this explicitly in Appendix E.
The design spaces are summarized for every environment family in Appendix A (Table 2) and described in full, including how to build a design, in the documentation released with the code.
A unified approach to modelling design spaces beyond vectors and fixed topologies (Q2) remains an open and interesting question. Graph-based representations have been explored before: encoding a robot's morphology as a graph whose nodes and edges carry link geometry, material properties and actuator parameters, then processing these representations with graph neural networks. It remains open whether such representations make co-design more efficient than flat vectors in general, or whether other representations would better support embodiment optimization and policy learning. Since the embodiment space is a standard Gymnasium Space, novel representations can be plugged in and tested; we provide flat vectors as the default because they are the most appropriate starting point now, given how difficult it is to foresee which structures users will want to try.
Regarding robot co-design: while robotics domains constitute a large part of the suite (the Native robotics families and the MuJoCo ports), we hope the non-robotic families add value by testing whether conclusions transfer beyond robots. Indeed, they already reveal some domain-specific winners (Section 6).
Q1: Multi-Robot Truck Unloadingz
That is a fair observation and our approach does not fully leverage the advantages of multi-agent co-design. The Multi-Robot Truck Unloading task allows for richer multi-agent co-design formulations (and the environment does not preclude these in principle) than the one we adopt as a baseline.
Our current formulation was a deliberate choice for the purposes of a benchmark, for three reasons: (i) it keeps the interface identical across all environments, so every co-design method in the benchmark runs without modification; (ii) it separates the co-design problem from well-known multi-agent RL difficulties, such as credit assignment, non-stationarity and communication, which would otherwise confound the comparison between co-design algorithms; and (iii) a centralized controller provides a natural reference against which decentralized formulations can be compared in future work.
More broadly, our primary contribution is a benchmark, not an optimal solution to each task. As in established RL benchmarks, such as the Arcade Learning Environment [6] and the DeepMind Control Suite [7], our baselines are intended as a starting point for comparison rather than a comprehensive documentation of the state of the art or an upper bound on what is achievable. Formulations that fully exploit multi-agent co-design in this task are precisely the kind of research we hope the benchmark will enable.
Thank you once more for the care you put into this review. Your challenge to measure the benefit of co-optimization carefully has shaped the large part of the revision; the new experiments based on your suggestions now make that benefit, and its current limits, more concrete and visible. We hope the new experiments and clarifications address your concerns, and we welcome any further discussion.
Response to Reviewer ecMz (1/3)
Thank you for your detailed review. We appreciate your recognition that the benchmark is "a real infrastructure contribution", and that it "broaden[s] evaluation beyond locomotion". We fully share your view that a Co-design benchmark should measure the benefit of coordinated optimization itself, and the revision now does so directly (W2 below).
You are right, and we are sorry for these oversights. All of them are corrected in the revised PDF:
The LLM table is now complete (Table 14). This exploratory experiment is intended to demonstrate what a very low-budget optimization approach with an LLM in the loop can achieve. We believe a low-cost experiment is still a useful investigation, because the cost of frontier models can make more ambitious comparisons financially prohibitive. Our benchmark focuses on wider participation and inclusion, as all of our environments and presets can be run on a single GPU. Our Gemma explorations were conducted within the free Google API tier, and as such represent an exploratory low-budget LLM experiment; we now label the study specifically as low-budget. We welcome suggestions on how to demonstrate the power of LLMs for co-design within reasonable budgets, to keep the benchmark open and accessible to a wide range of researchers.
We fully agree, so we have run two control settings that isolate the value of policy-only and design-only optimization (App. C.2, Tables 6–7). For policy-only optimization, we employ PPO with the fixed base embodiment of the environment, and for design-only optimization we initialize a random policy and perform pure design-space exploration. A random policy cannot deliver competitive performance on some environments (e.g., robotic control), but there is a subset of our presets on which a random policy may do something meaningful (e.g., Pokémon). Hence, the design-only control was run on those 13 presets. The results measure the design gain directly:
Note that fixed-design PPO solves a strictly smaller problem with a mature method, while co-design methods must search the joint space of designs and policies under the same budget, using far less mature methods. Since fixed-design policy learning is a special case of co-design, the best achievable co-design return is never lower (Proposition 4, Section 3). Hence, a fixed-design policy coming out ahead most likely reflects how hard and how young co-optimization is, not a general lack of benefit from co-design. Against this, the wins on 13 of 22 presets are encouraging.
The soft-body and shape-changing environments, which the reviewer noted were missing from the scoreboard, are now part of both the main scoreboard and the controls. The table shows these four presets; the results for all 22 presets with controls are in Tables 6-7 (App. C.2). Controls for the remaining 13 presets of the scoreboard are currently running, and we will add them to the paper as soon as they finish rather than delay our response:
These measured gains - positive on some presets and negative on others - provide the empirical counterpart to the existence results of Section 3 and Appendix F, and show that current methods, still young compared with policy-only RL, do not yet reliably convert design freedom into performance.
Furthermore, our released code contains documentation outlining the design space for every environment family (layout, bounds and how to build a design), the observations, actions and rewards, and the runnable examples. Appendix A now summarizes it in a table that gives, for every family, the design space, the rule for rejecting designs, the return range and the episode length (Table 2); the normalization anchors are in Table 3.