acceptodds
Under review as a conference paper at ICLR 2027

Probe-Diagnose-Adapt: An Evidence-Driven Embodied Research Loop for Robot Co-Design

Abstract

Robot co-design is costly because evaluating a morphology typically requires training a controller for it. Each trial, however, reveals more than a task return: it shows how the body moves and interacts with its surroundings. How can these observations guide the next design change? We introduce Probe-Diagnose-Adapt (PDA), an LLM-guided framework that organizes robot co-design as an embodied research loop. PDA trains a fresh controller for each candidate under the same training budget and summarizes its behavior through module-level probes fixed after calibration. Drawing on this evidence and past trials, the LLM identifies a capability to investigate, proposes a feasible body edit, and predicts a measurable change before testing. Each experiment assesses task performance and the predicted change separately: performance guides morphology selection, while the prediction check informs subsequent reasoning. The resulting experimental memory preserves exact edits and both outcomes, allowing successful, rejected, and inconclusive trials to inform later proposals. Under matched per-body training budgets, PDA achieves the highest mean search performance at 53 morphologies on all six Evolution Gym tasks. On five tasks, these returns are competitive with the endpoints of longer baseline searches. Controlled ablations show complementary gains from behavioral evidence, module-level localization, and experimental memory. On two articulated-robot tasks in UNIMAL, the same loop improves on initial designs, supporting applicability beyond voxel-based morphologies. By linking edits to prospective predictions and measured outcomes, PDA takes a step toward embodied auto research through budgeted experimentation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Co-Design, Embodied Co-Design, Large Models, Research Loop, Morphology, Embodied Experiment

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cXTWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Probe–Diagnose–Adapt (PDA), an LLM-guided robot co-design framework that uses behavioral evidence and experimental memory to guide morphology modifications through testable predictions and iterative evaluation. Experiments on six Evolution Gym tasks show improved search performance over existing baselines under matched morphology-evaluation budgets, while additional results on two UNIMAL tasks demonstrate applicability to articulated robots.

Critical strengths
  1. The paper is well-motivated. The use of behavioral evidence beyond task returns provides additional information for guiding morphology search.
  2. The evaluation covers diverse task types, including soft voxel-based robots and articulated rigid-body robots, supporting the applicability of the method across different robot representations.
  3. PDA achieves the highest mean performance among the compared methods on all six Evolution Gym tasks at the common evaluation budget.
Critical weaknesses
  1. Efficiency is measured by morphology evaluations, without LLM costs or wall-clock time. It would be helpful to report these costs and consider the parallelism available to population-based baselines.
  2. Limited LLM generalization. All experiments use DeepSeek-Flash. Testing additional LLMs or unseen tasks would help assess backbone dependence and potential reliance on prior knowledge of Evolution Gym.
Questions for authors

A key distinction from Auto-Robotist is the prediction and verification of behavioral changes, but this mechanism is not independently ablated. Could the authors provide an ablation that removes prospective prediction and verification while retaining the same behavioral evidence and memory?

5 comments
AuthorsOpenReview ↗

Response to Reviewer cXTW [1/3]

Thank you for encouraging us to connect PDA's performance to its practical cost and to test its dependence on the LLM backbone and prediction mechanism. Your questions really helped us distinguish controller-training efficiency from wall-clock efficiency and make the mechanism comparisons more direct.

[Weakness 1]: LLM costs, wall-clock time, and population parallelism.

[A1]: First, we report the LLM usage and end-to-end wall-clock cost of PDA. Morphology evaluations and wall-clock time capture different costs, and your comment prompted us to report both. The table below covers all 6 tasks at 53 morphology evaluations and 512K PPO steps per body (Sec. 4.6; App. N.1, Table 25).

TaskInput tokens (M)Output tokens (M)LLM-call wall-clock (h)End-to-end wall-clock (h)
Walker-v07.1512.7993.9125.90
Climber-v07.4372.7394.0626.77
UpStepper-v09.0003.2144.6029.38
Pusher-v07.7072.8254.1827.49
Carrier-v08.2403.0295.0528.58
Thrower-v010.0903.3745.9132.30

Each row is the measured task-level mean across 6 independent searches. End-to-end wall-clock time includes initialization, controller training, evaluation, and LLM calls through the final search evaluation. Note that the LLM-call column is included in that total.

Second, we compare how many morphology evaluations and how much wall-clock time each method needs to reach the same performance target. For each task, this target is the strongest baseline's mean return at 53 morphology evaluations. This comparison allows population-based baselines 4 concurrent candidate trainings, while PDA remains sequential. Each entry reports median morphology evaluations / wall-clock time (h) to reach the target return, and NR means the target was not reached within the available records (App. N.2, Table 26).

TaskTarget returnPDA, sequentialGA, 4 workersLASeR, 4 workersAuto-Robotist, 4 workers
Walker8.4746/3.6 h120/13.5 h46/5.3 h120/14.9 h
Climber0.64510/5.0 h446/51.2 h158/19.2 hNR
UpStepper3.13227/14.7 h225/28.5 h63/8.1 h450/59.9 h
Pusher6.98812/6.2 h235/27.4 h51/6.3 h250/31.5 h
Carrier7.41611/6.4 h260/31.6 h46/5.7 h200/25.3 h
Thrower1.45328/17.6 h150/20.5 h45/6.4 h75/10.7 h

The table makes the trade-off clear: among target-reaching searches, PDA requires the fewest morphology evaluations on all 6 tasks, but population parallelism can shorten wall-clock time. For example, LASeR is faster in the UpStepper, Carrier, and Thrower comparisons. We recognize this practical advantage.

PDA’s value is therefore in reducing costly controller-training trials, rather than always finishing sooner. With 512K PPO steps per body, each avoided controller training saves 512K training interactions; parallel execution does not eliminate those interactions. Additional reasoning can be worthwhile when controller training and candidate testing are the limiting resources. Although these results are from simulation, this distinction also motivates future applications involving costly physical interaction.

AuthorsOpenReview ↗

Response to Reviewer cXTW [2/3]

[Weakness 2]: LLM backbone dependence and benchmark knowledge

[A2]: We followed your suggestion to test additional LLMs on UpStepper-v0: all four added backbones improve designs during online search. We used the same 53-evaluation budget, with 6 different searches per added backbone. Improvement is the mean within-search gain and final performance is mean +/- across-search SD (Sec. 4.6; App. L.2, Table 22 and Fig. 11).

LLMInitial performanceFinal performanceImprovement
DeepSeek-Flash1.4554.663 +/- 1.087+3.208
GLM-5.3-Flash1.8204.930 +/- 0.231+3.109
Qwen3.8-Flash1.5003.898 +/- 0.060+2.399
Gemini 3.8 Flash1.4523.766 +/- 0.885+2.313
GPT-6 Luna1.4173.711 +/- 0.618+2.293

For a quick visual comparison: PDA's search curves with different LLM backbones.

GLM has the highest final mean, 4.930, while DeepSeek has the largest mean gain, 3.208. The other backbones also make online gains of 2.293-2.399. Performance varies across models, but all tested backbones improve their initial designs through online search. These results show that PDA's improvement on this task is not limited to DeepSeek-Flash. Since the models may already know Evolution Gym, testing unfamiliar tasks is an important next step.

AuthorsOpenReview ↗

Response to Reviewer cXTW [3/3]

[Question 1]: New ablation: Prediction and verification with factual memory retained

[A3]: We implemented exactly this ablation as PDA-NoPred: it retains current behavioral measurements and factual histories of bodies, edits, returns, and measured changes, but removes directional predictions and their assessments. Initialization, controller budgets, retrieval, and selection rules are shared with PDA-R, the full-information reference under this common ablation protocol. The table below summarizes 6 independent searches per task and variant, paired by initialization, after 20 online edits. Values are mean +/- across-search SD (Sec. 4.3, Fig. 4; App. I.3, Table 11).

TaskPDA-RPDA-NoPredRelative mean gain
UpStepper-v03.611 +/- 0.3282.481 +/- 0.03745.6%
Pusher-v09.852 +/- 0.1998.965 +/- 0.7959.9%

With behavioral measurements and factual history available to both methods, PDA-R achieves higher mean returns than PDA-NoPred on both tasks. Search trajectories: EvoGym ablation curves. The corresponding NoPred comparison on UNIMAL also favors PDA in mean: 25.9% on Incline and 30.4% on Push Box Incline (Sec. 4.5, Fig. 5; App. K.4, Table 20). For a visual comparison: UNIMAL search curves.

Furthermore, we introduced PDA-Facts to test what information should be carried forward between trials. It retains behavioral measurements and still generates and checks predictions for the current trial, but provides subsequent decisions with only factual history: bodies, edits, returns, and measured changes. PDA-R additionally retains past hypotheses, predictions, and their assessments. PDA-R's higher mean returns support reusing these records to guide later edits, beyond generating and checking predictions within each trial (Sec. 4.3; App. I.3–I.4).

For a quick, systematic comparison: Anonymous ablation overview. It brings together the original and added ablations, their information matrices, performance curves, and compact examples of the actual decision inputs. We hope this makes it easy to see both what each ablation isolates and which conclusions the results support.

Thank you again for these suggestions. They helped us test whether predictions and their assessments improve later design decisions, and show the computational cost alongside the performance gains.

Reviewer_aVmqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Probe–Diagnose–Adapt (PDA). Probe trains a controller from scratch for each candidate morphology and constructs evidence using module-level probes that are fixed after initialization, trajectory summaries, and static geometry. Diagnose uses this evidence to formulate a hypothesis about a capability to be tested. Adapt proposes valid morphology edits and specifies the expected direction of change in a metric before the trial. The system separately assesses improvements in return and whether the prediction is supported, selects designs based on return, and stores the edits and both types of outcomes from successful, unsuccessful, and inconclusive trials in memory. The primary evaluation covers six tasks in Evolution Gym. With 53 evaluations, including three initial morphologies, PDA achieves the highest mean search performance on all six tasks. On five tasks, its performance is competitive with the endpoints of longer baseline searches.

Critical strengths
  1. The method yields substantial performance gains under a limited budget, and the comparison controls for the main training-related confounders. Its advantages span both locomotion and manipulation tasks, rather than being explained solely by a single illustrative case.
  2. Separating performance-based selection from verification of explanations, while retaining traceable counterevidence, constitutes a useful methodological design. Fixed measurement definitions, explicit sources and spatial and temporal scopes, and the recording of edits and predictions before each trial make the process easier to inspect than feedback based solely on return. They also avoid treating a high score as direct evidence that an explanation is correct.
Critical weaknesses
  1. The ablations do not isolate the contributions of prospective predictions and prediction verification, which limits, to some extent, the evidence supporting the method's core novelty. In the experiments, the transition from PDA-S to PDA-G simultaneously adds behavioral measurements and prediction targets, while the transition from PDA-L to PDA-R simultaneously adds hypotheses, predictions, historical measurement changes, assessments, and explanations. All variants already retain histories of morphologies, edits, and scores; therefore, the latter comparison demonstrates the overall value of richer experimental records. The gains may also arise from additional behavioral information or longer context, and cannot yet be attributed solely to committing to predictions in advance and verifying them.
  2. The reliability of prediction support and the scope of the performance advantages still lack conclusive validation. For example, Figure 3 does not provide paired differences or their intervals, and UNIMAL is compared only against initialization. The current evidence supports search gains under a particular budget, but is not yet sufficient to establish consistent advantages across all components, tasks, and platforms.
Questions for authors
  1. With the same local behavioral evidence, retrieved records, context budget, and editing rules, are there controls that remove only prospective predictions or only prediction-verification feedback? If not, could the conclusions from the existing ablations be explicitly limited to the gains from the combination of information, rather than attributing them specifically to prediction verification?
  2. Could you clarify whether the Thrower results in Table 1 support consistently outperforming the strongest baseline under the same budget? The reported means and SDs are insufficient to determine whether smaller component-level gains are reliable.
  3. Was the final output reevaluated using evaluation seeds that were not used during search, and were the key selected morphologies evaluated through repeated controller training? If the Evolution Gym environment and policy produce identical trajectories across different scoring seeds, please clarify what sources of randomness the seven-episode panel actually covers.
5 comments
AuthorsOpenReview ↗

Response to Reviewer aVmq [1/4]

Thank you for the thoughtful review. Your questions really helped us distinguish three issues: whether prediction-related records improve search, whether individual edits remain effective after controller retraining, and whether selected final bodies retain their performance under independent reevaluation. We added targeted comparisons for each and explain the results below.

[Weakness 1 / Question 1]: Mechanism attribution with behavioral measurements retained

[A1]: We added ablations that retain current behavioral measurements and factual histories of bodies, edits, returns, and measured changes. Unlike the original cumulative S/G/L/R study, these comparisons change how predictions and their assessments are used while holding initialization, controller budgets, retrieval, and selection rules constant. All variants use the same maximum input length, but their actual prompts are not equal in length (Sec. 4.3; App. I.3, Table 11).

PDA-R is the full-information reference within this shared ablation protocol. All variants edit the current body and retrieve the latest three trials; the main PDA searches instead use context-ranked retrieval and allow follow-up edits from retained bodies (Sec. 4.3; App. I.2–I.3).

The table summarizes six independent searches per task and variant, paired by initialization, after 20 online edits. Values are mean +/- across-search SD.

VariantWhat the variant doesUpStepper-v0Pusher-v0
PDA-RMakes and checks predictions; keeps past reasoning, predictions, assessments, and lessons.3.611 +/- 0.3289.852 +/- 0.199
PDA-NoPredUses factual history without making or assessing explicit predictions.2.481 +/- 0.0378.965 +/- 0.795
PDA-FactsMakes and checks current predictions, but passes only factual history to later decisions.2.394 +/- 0.3839.167 +/- 0.125
PDA-NoCheckLater decisions receive past reasoning, predictions, and measured outcomes, but no explicit assessments or lessons.1.927 +/- 0.5788.132 +/- 0.893

Search trajectories: EvoGym ablation curves. These comparisons are now shown in the main text (Sec. 4.3, Fig. 4).

First, PDA-R improves mean return over NoPred by 45.6% on UpStepper and 9.9% on Pusher. Both receive behavioral measurements and factual history. R also makes explicit predictions, checks them against the observed outcomes, and retains these records for later decisions. This comparison supports using these steps together, rather than identifying the effect of prediction or checking alone.

Second, we test what should be remembered from earlier trials. Both R and Facts make and check predictions for the current trial. However, Facts passes only bodies, edits, returns, and measured changes to later decisions. R also remembers why an edit was proposed, what it was expected to change, and whether the measurements supported that expectation. R has higher mean returns on both tasks, whereas Facts does not consistently outperform NoPred. These results support keeping past reasoning, predictions, and assessments together for later decisions.

Third, we test whether explicit assessments and lessons help later decisions. NoCheck retains predictions and measured outcomes, so the LLM can still infer whether they agree. R additionally provides the explicit assessment and the lessons drawn from it, and has higher mean returns on both tasks. This comparison evaluates assessments and their lessons together, rather than the assessment label alone.

For a quick, systematic comparison: Anonymous ablation overview. It separates the original and added studies and brings together information matrices, performance curves, and compact examples of the actual R and Facts inputs. We hope this makes both the ablations and their conclusions easier to inspect.

AuthorsOpenReview ↗

Response to Reviewer aVmq [2/4]

[Weakness 2]: Paired differences and intervals for Figure 3

Thank you for pointing out that the SD bands in Figure 3 do not quantify uncertainty in the differences between variants. We now report paired differences in best-so-far return after 20 online edits, corresponding to 23 morphology evaluations including initialization. For each comparison, we calculate the return difference within each of six shared-initialization pairs. The original and added studies are analyzed separately. Each cell reports the mean paired difference [95% paired t confidence interval], without adjustment for multiple comparisons (App. I.4, Table 12).

The added ablations support the value of using predictions, assessments, and their history together. R has higher endpoint returns than NoPred, Facts, and NoCheck on both tasks, with the corresponding intervals above zero. As explained above, these comparisons test different parts of that process, rather than attributing the full gain to prediction alone.

[Weakness 2 / Question 2]: The Thrower comparison and uncertainty

[A2]: On Thrower, PDA has an early-budget mean advantage, while LASeR improves further with a larger budget. At 53 evaluations, PDA obtains 1.645 +/- 0.464 versus 1.453 +/- 0.176 for LASeR, across 6 independent searches per method. The mean difference is +0.192, with a search-level 95% Welch confidence interval of [-0.296, 0.680]. The interval includes zero, so these runs do not establish a reliable performance advantage over LASeR at the same budget. The mean gap is small relative to the variation across searches. Our Thrower analysis also suggests why further improvement can be difficult: proposals frequently make small local edits and reuse a narrow set of progress measurements. In one recorded trial, post-peak object velocity increases as predicted, yet displacement and return both decrease. This illustrates how predicting a measured change correctly can still miss what makes a better throwing design (App. J.1–J.3; Sec. 5).

At 500 evaluations, LASeR reaches 3.262 +/- 0.127 (Sec. 4.2, Table 1; App. M.4).

[Weakness 2]: Search baselines and ablations on UNIMAL

[A3]: Both UNIMAL tasks now include search baselines and a NoPred ablation that retains behavioral measurements and factual history. PDA, NoPred, and random-edit search share initialization; all methods use the same legal design space and per-body controller-training protocol. The table reports best-so-far return, mean +/- across-search SD, from 6 independent searches per method and task (Sec. 4.5, Fig. 5; App. K.4, Table 20).

MethodMorphology evaluationsInclinePush Box Incline
PDA535289.4 +/- 285.81543.3 +/- 91.0
PDA-NoPred534200.0 +/- 571.51183.3 +/- 62.4
Random-edit532864.7 +/- 377.5873.1 +/- 37.6
DERL-style1003930.0 +/- 278.01092.0 +/- 144.9

For a quick visual comparison: UNIMAL search curves.

At 53 evaluations, PDA exceeds NoPred in mean by 25.9% on Incline and 30.4% on Push Box Incline, and also achieves higher mean returns than random-edit search at the same budget and DERL-style search at 100 evaluations. Thus, the additions test both search quality and the combined prediction-and-memory process in a second body representation. Our budget-scaled DERL-style baseline uses a population of 10, three-member tournaments for parent selection, and removal of the lowest-scoring member during replacement (App. K.4).

AuthorsOpenReview ↗

Response to Reviewer aVmq [3/4]

[Question 3, Part 1]: Training randomness and fixed-edit replication

[A4]: For six selected morphology edits, we retrained controllers for both the body before the edit and the body after it. We ran five PPO trainings per body at 512K steps each, totaling 60 fresh controller trainings. Bodies, probes, predicted directions, and assessment rules were held fixed (Sec. 4.6; App. M.1, Table 23).

The return difference is the edited body's return minus the original body's return after retraining. The table reports its mean +/- sample SD and the number of repetitions in which return improves or the prediction is supported.

Fixed editReturn difference after retrainingRepetitions with higher returnRepetitions supporting the prediction
Climber, trial 17+2.335 +/- 2.2623/54/5
Climber, trial 18-0.772 +/- 3.2363/52/5
Carrier, trial 1+2.864 +/- 2.3924/50/5
Carrier, trial 3+0.086 +/- 2.3004/54/5
Thrower, trial 14+0.051 +/- 0.1023/53/5
Thrower, trial 16-0.011 +/- 0.3463/52/5

Some edits still improve return after retraining, while others do not. For Climber trial 17, the edited body has a higher mean return than the original body; for trial 18, it has a lower mean return. The Thrower return differences are close to zero. Carrier trial 1 is particularly informative: return improves in four repetitions, while the prediction is supported in none of the five. An edit can therefore improve task performance without supporting its accompanying prediction.

This is why we evaluate performance and prediction separately. The ablations test whether these records improve subsequent search. The retraining study tests whether the same body edit again improves return and produces the predicted behavior after learning a new controller.

[Question 3, Part 2]: The seven-episode evaluation panel

[A5]: You are right that seven evaluation episodes do not measure PPO training uncertainty. Each trained controller is evaluated seven times with different environment seeds, using deterministic actions. In the replication study, all seven returns are identical within each of the 60 controllers. We therefore use fresh PPO training to measure training variability and independent searches to measure search variability. The seven episodes are evaluations of one trained body-controller pair, not seven independent training repetitions (App. M.2).

Reviewer_n8QROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose a new framework for the LLM-driven automatic design of robots.

Critical strengths

The method allows detailed behavioral information to guide design, rather than a single fitness score.

Critical weaknesses

Experiments involve extremely small design spaces. Planar soft bodies were confined within a 5x5 grid. In the rigid bodies, almost all adjustable parameters are selected from predefined discrete values. For example, limb length can only take three values of 0.2, 0.3, and 0.4 meters (Table 17). Body shape and variation operators are also restricted. For the rigid bodies, two incline tasks are presented without search baselines or ablations. As a result, the performance, scalability, and generalizability of the method remain unclear.

Rigid body experiments only compared PDA with its initial performance. This reference came from the best of three initial designs in each run. No alternative methods were evaluated.

The benefit of prediction and verification also remains unclear due to poor experimental design. Several types of information were added together in each ablation comparison. A better control experiment would have used the same current and historical measurements without explicit predictions or prediction assessments.

Questions for authors

None

4 comments
AuthorsOpenReview ↗

Response to Reviewer n8QR [1/3]

Thank you for identifying the comparisons needed to assess PDA more convincingly. We took these concerns seriously and added experiments in three directions: a larger design space, search baselines on articulated robots, and prediction ablations that retain behavioral measurements. Below, we explain what each comparison adds and the scope of the conclusions it supports.

[Weakness 1]: Search-space size and scalability

[A1]: In response to this concern, we tested PDA on a 10 x 10 Walker-v0 design space, where it retains its search advantage. Enlarging the grid from 5 x 5 increases the available cell positions from 25 to 100. We compared PDA with all five search baselines from the paper in this larger space. The table below summarizes results of 6 independent searches per method. Values are mean +/- across-search SD (Sec. 4.4; App. L.1, Table 21).

MethodBest recorded return within 53 evaluationsFinal recorded return
PDA10.389 +/- 0.03910.389 +/- 0.039
GA3.046 +/- 1.3946.840 +/- 0.562
BO6.836 +/- 0.2027.769 +/- 0.486
CPPN-NEAT7.597 +/- 0.6219.742 +/- 0.834
LASeR8.403 +/- 0.2769.906 +/- 0.742
Auto-Robotist2.565 +/- 0.6857.425 +/- 1.375

The final column reports the best return recorded within each method's evaluation budget: 53 morphology evaluations for PDA and 300 for each baseline. In this experiment, PDA reaches 10.389 +/- 0.039 after an average of only 14 online edits. Its mean is 23.6% higher than the strongest baseline within 53 evaluations and also exceeds the final recorded means of the longer baseline searches.

For a quick visual comparison: 10 x 10 search curves and PDA design sequence. The illustrated body spans eight columns and seven rows, showing that PDA uses the expanded space rather than remaining inside the original 5 x 5 envelope (App. L.1, Figs. 8–9).

For UNIMAL, we would like to clarify that the search varies both topology and physical parameters: it can add or remove limbs in a 2-10-limb kinematic tree and change limb lengths, joint degrees of freedom, joint ranges, gearing, and density. The discrete limb lengths of 0.2, 0.3, and 0.4 m follow Gupta et al., Appendix A. These choices still define a constrained design space, as you noted, but vary topology as well as dimensions. The comparisons below test how effectively PDA searches this space (App. K.2, Table 18).

AuthorsOpenReview ↗

Response to Reviewer n8QR [2/3]

[Weakness 2]: Search baselines and ablations on UNIMAL

[A2]: We added two search baselines and a prediction ablation to both Incline and Push Box Incline. We agree that improvement over initialization alone does not establish a search advantage. Legal random-edit search and DERL-style population search now provide alternative search baselines. PDA-NoPred addresses the complementary mechanism question: it retains current behavioral measurements and factual histories of bodies, edits, returns, and measured changes, but removes directional predictions and their assessments. This tests whether the prediction mechanism helps when behavioral evidence remains available.

To make the comparison fair, PDA, NoPred, and random-edit search share initialization, and all methods use the same legal design space and per-body controller-training protocol. Initialization evaluations count toward the budget. Random-edit search proposes changes to the current body without online LLM calls and uses the same return-based acceptance test as PDA. DERL-style uses a population of 10, selects parents through three-member tournaments, and removes the lowest-scoring population member during replacement. We report its results at a near-matched 50-evaluation checkpoint and at 100 evaluations to examine its progress with a larger budget. It is a budget-scaled evolutionary comparator, not a reproduction of DERL's original large-scale setup. The table reports best-so-far return, expressed as mean +/- across-search SD, from 6 independent searches per method and task (Sec. 4.5; App. K.3–K.4, Tables 19–20).

MethodEvaluation budget / checkpointInclinePush Box Incline
PDA535289.4 +/- 285.81543.3 +/- 91.0
PDA-NoPred534200.0 +/- 571.51183.3 +/- 62.4
Random-edit532864.7 +/- 377.5873.1 +/- 37.6
DERL-style503116.7 +/- 91.0790.3 +/- 178.7
DERL-style1003930.0 +/- 278.01092.0 +/- 144.9

For a quick visual comparison: UNIMAL search curves. The curves are also included in the revised main text (Sec. 4.5, Fig. 5).

PDA at 53 evaluations exceeds DERL-style search in mean at both checkpoints, including gains of 34.6% on Incline and 41.3% on Push Box Incline over its 100-evaluation result. The random-edit comparison adds a shared-initialization reference, while NoPred tests the prediction mechanism. Together with the larger-grid EvoGym experiment, these results support efficient search in the tested spaces and applicability across two body representations. Building on these findings, we are keen to investigate whether these benefits extend to unseen tasks and more complex design spaces, an important direction highlighted by your feedback.

AuthorsOpenReview ↗

Response to Reviewer n8QR [3/3]

[Weakness 3]: Ablations: prediction and verification with factual evidence retained

[A3]: We implemented the ablation you proposed as PDA-NoPred: it retains current behavioral measurements and factual histories of bodies, edits, returns, and measured changes, but removes directional predictions and their assessments. We also added PDA-Facts and PDA-NoCheck to distinguish historical reuse from explicit assessment feedback. All four variants retain those measurements and factual records, with shared initialization, controller budgets, retrieval, and selection rules.

Here, PDA-R is the full-information reference within the shared ablation protocol: all variants edit the current body and retrieve the latest three trials. The main PDA searches instead use context-ranked retrieval and allow follow-up edits from retained bodies. The original and added ablation studies use separate search runs and are analyzed separately (Sec. 4.3; App. I.2–I.3).

The table below summarizes 6 independent searches per task and variant, paired by initialization, after 20 online edits. Values are mean +/- across-search SD (Sec. 4.3, Fig. 4; App. I.3, Table 11).

VariantWhat the variant doesQuestion tested against RUpStepper-v0Pusher-v0
PDA-RMakes and checks predictions; keeps past reasoning, predictions, assessments, and lessons.Reference for the three ablations below.3.611 +/- 0.3289.852 +/- 0.199
PDA-NoPredUses factual history without making or assessing explicit predictions.Do predictions and checks help beyond measurements and factual history?2.481 +/- 0.0378.965 +/- 0.795
PDA-FactsMakes and checks current predictions, but passes only factual history to later decisions.Does keeping past reasoning, predictions, and assessments help later decisions?2.394 +/- 0.3839.167 +/- 0.125
PDA-NoCheckLater decisions receive past reasoning, predictions, and measured outcomes, but no explicit assessments or lessons.Do explicit assessments and their lessons help later decisions?1.927 +/- 0.5788.132 +/- 0.893

Search trajectories: EvoGym ablation curves.

The comparisons answer three distinct questions:

  • Do predictions and checks help when measurements are already available? Both R and NoPred receive behavioral measurements and factual history. R also makes explicit predictions, checks them against the observed outcomes, and retains these records for later decisions. Its mean return is 45.6% higher on UpStepper and 9.9% higher on Pusher. This directly answers your proposed comparison and supports using these steps together.

  • What should be remembered from earlier trials? Both R and Facts make and check predictions for the current trial. However, Facts passes only bodies, edits, returns, and measured changes to later decisions. R also remembers why an edit was proposed, what it was expected to change, and whether the measurements supported that expectation. R achieves higher mean returns on both tasks, whereas Facts does not consistently outperform NoPred. These results support retaining the reasoning, predictions, and assessments from past trials, beyond making predictions only for the current trial.

  • Do explicit assessments and lessons help later decisions? NoCheck retains predictions and measured outcomes, so the LLM can still infer whether they agree. What it does not receive is the explicit assessment and the lessons drawn from it. R has higher mean returns on both tasks, supporting the usefulness of providing this feedback to later decisions. This comparison evaluates assessments and their lessons together, rather than the assessment label alone.

These results favor using predictions, outcome assessments, and memory together. Furthermore, at the reported UNIMAL budgets, PDA also exceeds NoPred in mean by 25.9% on Incline and 30.4% on Push Box Incline, extending the comparison to articulated bodies.

For a quick, systematic comparison: Anonymous ablation overview. It separates the original S/G/L/R study of progressively richer evidence from these targeted ablations, with information matrices, curves, and compact examples of actual decision inputs. We hope this makes the experimental differences and their conclusions easy to inspect.

Thank you again for these suggestions. They helped us replace missing comparisons with direct tests and define the scope of our conclusions more clearly. We have included the central results in the Experiments section and the supporting settings and analyses in the appendix, so readers can assess both PDA's search efficiency and the contribution of experimental memory (Secs. 4.3–4.5; Apps. I, K, and L).

We sincerely hope to address your concerns and questions regarding our paper.

AuthorsOpenReview ↗

Follow-up on the prediction ablation

Dear Reviewer n8QR,

Thank you again for suggesting a comparison that keeps the same current and historical measurements but removes explicit predictions and their assessments. We added that comparison as PDA-NoPred. After 20 edits, PDA-R achieved a higher mean return on both UpStepper and Pusher (Fig. 4). We also added search baselines on both UNIMAL tasks and tested a larger 10 × 10 EvoGym grid (Secs. 4.4–4.5).

We would welcome your thoughts on whether these comparisons address the concerns you raised.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Response

We thank all three reviewers for their careful reading and constructive feedback. Their main concerns were whether predictions help beyond behavioral measurements, whether the search advantages extend beyond the original design space, and how training randomness, LLM choice, and computational cost affect the results. We added direct comparisons to address these questions, and detailed results are provided in the individual replies.

Inspired by the reviewers, what did we add during the rebuttal phase?

1. Tested predictions while retaining measurements and factual history. PDA-NoPred retains current measurements and factual history but removes predictions and their assessments. Across six paired searches per task, PDA-R exceeds PDA-NoPred in mean by 45.6% on UpStepper and 9.9% on Pusher after 20 online edits. PDA-Facts makes and checks current predictions, but passes only factual history to later decisions. PDA-NoCheck retains past reasoning, predictions, and measured outcomes, but withholds explicit assessments and their lessons. Together, the results support using predictions, assessments, and past records in combination. Our anonymous ablation overview brings the comparisons, curves, and actual input examples together for quick inspection.

2. Expanded the design space and added articulated-body baselines. On a 10 x 10 design space of Walker-v0, PDA's mean return is 23.6% higher than the strongest baseline within 53 evaluations (Larger-grid results). On both UNIMAL tasks, PDA at 53 evaluations exceeds NoPred and random-edit search at the same budget, and DERL-style search at 100, in mean return (UNIMAL results).

3. Checked performance after controller retraining. We retrained controllers for bodies before and after selected edits; some edits did not consistently improve return. We also kept each final body unchanged and trained four new controllers with different PPO seeds, evaluating them on seeds unused for selection. Across six tasks, the final bodies' mean returns were 1.1% lower to 5.3% higher than during search.

4. Tested four additional LLM backbones (GPT/Qwen/GLM/Gemini). All improve their initial designs through online search on UpStepper-v0, showing that improvement on this task is not limited to DeepSeek-Flash (Cross-backbone results).

5. Reported costs and accounted for population parallelism. We provide six-task token and wall-clock costs and allow population baselines four concurrent candidate trainings in the common-target comparison. Among methods that reach the same target return, PDA requires fewer morphology evaluations, although parallel baselines can finish sooner.

Changes to the manuscript

We have updated the manuscript, with revisions marked in blue. The main changes are:

Added:

  • Prediction and memory ablations in the main text, with input definitions and paired difference intervals in the appendix (Sec. 4.3, Fig. 4; App. I.3–I.4, Tables 11–12).
  • Larger-grid and articulated-body search comparisons, with design examples (Secs. 4.4–4.5, Fig. 5; App. K.4–K.5 and L.1, Tables 20–21, Figs. 7–10).
  • Fixed-edit replication and independent final-body re-evaluation after fresh controller training (Sec. 4.6; App. M.1–M.3, Tables 23–24).
  • Additional LLM backbones and six-task token and wall-clock comparisons, including population parallelism (Sec. 4.6; App. L.2 and N, Tables 22 and 25–26, Fig. 11).

Clarified and streamlined:

  • How PDA-R differs from the main PDA search protocol, which information each ablation removes, and why the original and added runs are analyzed separately (Sec. 4.3; App. I.2–I.4).
  • Shared initialization, training budgets, and baseline selection and replacement rules for UNIMAL comparisons; the uncertainty of the Thrower advantage; and the scope of the conclusions. We also removed repeated protocol and result descriptions (Secs. 4.2 and 5; Apps. B.2, K.4, and M.4).

Value of Probe-Diagnose-Adapt

We took the reviewers' concerns seriously and used the additional experiments to test why PDA works, where its advantages hold, and whether its results persist under fresh controller training. These comparisons support PDA's central idea: each costly morphology trial can guide the next design through a prediction, a measured outcome, and a record of what was learned. In the tested settings, PDA achieves strong returns with fewer controller-training trials, while keeping performance improvement separate from whether a prediction was supported. We believe this is a practical contribution to robot co-design and embodied agents: making progress from a limited number of embodied experiments, including those that do not improve the design.

AuthorsOpenReview ↗

Closing summary and thanks

We thank the reviewers and Area Chair for their time and suggestions. Their comments led to three substantial additions to the revised paper:

  • Testing whether predictions help. Following the requested comparison, both variants retain current and historical behavioral measurements. The variant retaining explicit predictions and their assessments achieved 45.6% higher mean return on UpStepper and 9.9% higher on Pusher after 20 edits (Sec. 4.3, Fig. 4). The ablation overview makes these comparisons easy to inspect.
  • Testing larger designs and stronger search baselines. On 10 × 10 Walker, PDA achieved 23.6% higher mean return than the strongest baseline at 53 evaluations. On both UNIMAL tasks, PDA at 53 evaluations also achieved higher mean returns than random editing at 53 and DERL-style search at 100 (Secs. 4.4–4.5).
  • Checking performance after fresh training. Each selected final body was retrained four times with different PPO seeds and evaluated on held-out seeds. Across six tasks, mean returns changed by −1.1% to +5.3% relative to those recorded during search (App. M.3).

PDA aims to find better robot bodies with fewer expensive controller-training runs. Its contribution is to predict what a body edit should change, check what actually happened, and use that result to choose the next edit. We hope the revised experiments make the value of this approach clearer. Thank you again for considering our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.