acceptodds
Under review as a conference paper at ICLR 2027

The Effects Of Volumetric Morphology Encodings On Online Universal Control

Abstract

Universal control methods tackle the problem of learning to control diverse robot morphologies from online interactions, using reinforcement learning. Recent works have demonstrated that proper conditioning on morphological context is key to obtaining good performance across these incompatible Markov decision problems, i.e. problems with varying input and output spaces. In this work, we identify the robot's kinematic context as a particularly important portion of the overall contextual information. To provide the body shape portion of the kinematic context to universal controllers, existing methods have leveraged the strong assumption that robot bodies are composed of simplified geometric primitives and have directly included the descriptors of those shapes in the context vector. In this paper, we propose volumetric morphology encodings (VME). These novel representations capture the body shape and connectivity of the robot, and are learned during a pretraining phase, from a 3D robot body shape dataset that we collect. We use VME embeddings to improve the training of universal controllers, on challenging online universal control tasks. Even if we only alter the body shape portion of the kinematic portion of the overall robot context, we observe substantially higher performance than baseline methods, with higher gains in harder tasks. Furthermore, our approach demonstrates superior zero-shot transfer to unseen embodiments.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Generalization and multi-task learning, online universal control, locomotion, incompatible markov decision problems, robotics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_8jqTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Volumetric Morphology Encodings, which uses a pretrained 3D convolutional autoencoder to encode robot geometry into compact morphology representations for universal control. These embeddings replace hand-crafted descriptors in existing RL policies. Experiments on UNIMAL demonstrate improved control performance, particularly on challenging tasks, and better zero-shot generalization to unseen robot morphologies.

Critical strengths
  1. the idea is straightforward and the research question is importnat for crossembodiement robot learning.
  2. the method is novel and easy to understand.
  3. the improvement over the baselines is substantial
Critical weaknesses
  1. the task only compare with locomotion tasks, however, it is unclear how this can genearlize to other types of tasks like manipulation.
  2. the baselines compared here is very limited to robot morphology representation. It is important to compare with different way s of encoding the robot like urdf / grasphs.
Questions for authors

N/A

1 comment

Thank you for finding the question important, the method novel and the improvement substantial. Both of your concerns point to places where the paper should be clearer. In our revision, all changes are in blue.

W1: Only locomotion; what about manipulation? The paper does include a manipulation task. MVT (Manipulation in Variable Terrain) requires the robot to reach an object and push it t o a goal over uneven terrain. It was proposed with UNIMAL (Gupta et al., 2021) but dropped by later online UC work. VME's largest relative gains are on MVT. On UNIMAL (Fig. 4), success improves by +45% (+66%) over ModuMorph MVT (MVT(θ)) and by +15% (+53%) over MetaMorph MVT (MVT(θ)). On Mesh-UNIMAL (Fig. 8), the improvement is +179% and +341%. MVT success is sparse and varies between seeds, so no single MVT comparison against D+K is significant at 5 seeds. However, all four favour VME. And quantitatively, VME policies learn to "kick" the object (Fig. 7, App. B.2.1).

As for R_test, all methods are near zero, so we do not claim that the manipulation gains transfer to unseen morphologies.

We note that online UC over hundreds of morphologies has so far been studied almost only on locomotion. Manipulation with robot arms and grasping at this diversity would need a new benchmark and new methods, since grasp representations (UniGrasp, AdaGrasp, GET-Zero) consider a very limited number of robot gripper finger count (far from the |M|=200 afforded by UNIMAL).

W2: Compare with URDF / graph encodings. Thank you for this comment. We acknowledge that we were unclear in our baseline definitions. We will make this more explicit in our revision.

  • D+K is the URDF encoding. It is read directly from the robot description file (App. A.5), as in all prior online UC methods, and we test it in two state-of-the-art controllers.
  • Graphs. For all baselines and ablations, the robot graph already enters both the controller's attention and its context by a context element called body_parent. As for GNN-based online UC (NerveNet, SMP), this approach has been superseded by transformer-based online UC (Kurin et al., 2021; Gupta et al., 2022), which is why we build on the latter. But we note that VME is also graph-aware: a voxel grid is a regular graph over 3D space, and each VME token encodes its limb's joint region and the rest of the body, so connectivity is part of the learned representation. On ModularRL, the benchmark that was commonly used in the era of GNN-based online UC (App. B.3), VME gains +22.6% (+14.9%) over ModuMorph (MetaMorph) in the hard target-velocity regime and matches them on the easy one.
  • A learned URDF encoding (MatME). MatME is an autoencoder strategy applied to the URDF descriptors. Its encoding loses almost no information. Yet VME beats MatME in 20 out of 20 UNIMAL tasks (p<0.001). MatME is even below the raw descriptors (D+K) in 18 out of 20 tasks, while VME is above them in 19 out of 20. The format of the body-shape context matters, not only its information content, which is core to our contribution.
Robot setVME: CEVME: limbs perfect (same class)MatME: MSEMatME: limbs perfect (allowing 5% absolute error)
R_train0.0850 / 1,170 (0%)5e-51,170 / 1,170 (100%)
R_unseen0.08911 / 28,080 (0.04%)7e-526,474 / 28,080 (94.3%)
R_test0.6670 / 1,117 (0%)3e-2132 / 1,117 (11.8%)

We hope that this clarifies our experiments. If so, we would be grateful if you would consider raising your score, and we thank you again for your review.

Reviewer_y5ESOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how robot morphology representations affect online universal control. It proposes volumetric morphology encodings (VME), which use a pretrained convolutional autoencoder to compress voxel representations of individual limbs, their surrounding bodies, and joint regions. The resulting embeddings replace the body-shape portion of the context supplied to existing universal controllers. Experimental results generally show improved control performance, particularly on challenging tasks, while transfer to entirely unseen morphologies remains less consistent.

Critical strengths

A practical contribution that works across controller architectures.

Critical weaknesses

Limited evidence for reliable transfer to entirely new morphologies, especially manipulation. The paper shows useful training improvements and transfer gains on many locomotion settings. However, manipulation success on unseen robots remains very low and variable. This limits the breadth of the generalization claim.

Questions for authors

Which robot sets were used for encoder training and model selection? Were the morphologies used for transfer evaluation excluded?

Some typos: line 205: 'about about' line 269: 'save'

1 comment

Thank you for the positive assessment. Your question about which robots the encoder saw also showed us an ambiguity in App. A.6, which we have addressed in our revision (changes are in blue).

W: Transfer to new morphologies, especially manipulation. In our new revision, we now have a detailed statistical analysis section, of which we offer subset here:

Task groupR_unseen (6 variants)R_test
Unmodified27/30, p<0.0014/5, p=0.018
Pinned high velocity24/24, p<0.0014/4, p=0.003
θ30/30, p<0.0015/5, p=0.056
Pinned high velocity + θ24/24, p<0.0013/4, p=0.114
All tasks116/120, p<0.00117/20, p<0.001

On R_test, VME is higher in 17/20 tasks (e.g. +28% IN, +41% IN(3), +18% OB), with P(VME>D+K)=0.69 [0.61, 0.75]. And across all 140 transfer categories, VME is never significantly worse. As for transfer to entirely new morphologies (R_test), you are right that this is very difficult, and accordingly every method scores low there. However, within that regime, VME helps consistently: VME's median gain is +21% on R_unseen but +12% on R_test. Therefore, we have tangible evidence for our claim that VME improves controller generalization and transfer (supported by our results on both R_unseen and R_test).

Also, in practice, a simple way to improve VME on R_test would be to pretrain it on a broader dataset of shapes, allowing for stronger encoder generalization.

As for manipulation, we agree that it is a very difficult task regime. In fact, MVT was dropped by follow-up work, presumably because of its difficulty. However, we find that VME provides tangible improvements on this difficult regime; on R_unseen, VME beats D+K in 11/12 MVT cells (median +85%, e.g. Damping MVT(θ): 16.7 vs 4.4). However, seed variance is high, so this is not significant on its own (p=0.17). And on R_test, all methods are near zero (≤4.2%).

Overall, the weak performance on R_test and R_unseen is indicative of a zero-shot transfer problem that the online UC field has yet to solve; but it is a problem on which we make statistically significant progress with our contribution.

Q: Robot sets for encoder training and model selection.

  • Controller training: Only R_train.
  • Model selection: We base our controller hyperparameters (HPs) on the ModuMorph defaults, fixed for all tasks and methods. This is intentional: prior work tuned controller hyperparameters per-task, but we avoid doing this in order to show that VME's gains apply generally, without the need for HP tuning. As for the choice of the latent size (L=16) used by encoders, we chose it by looking at the diminishing returns on the reconstruction loss of R_train only (Fig. 12).
  • VME encoder training: trained only on R_train voxel grids. Our transfer experiments are thus zero-shot for both the encoder and the controller

Typos. Thank you for identifying these typos, which we have now fixed.

We hope the new statistical analysis, and the confirmation that the transfer experiments are fair, addresses your concerns. If so, we would be grateful if you would consider reflecting this in your score. We wish to thank you again for your questions and comments.

Reviewer_6vtMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies online universal control for morphologically diverse robots and argues that the kinematic/body-shape portion of the morphology context is especially important. The authors first conduct a motivating context ablation on the variable-terrain task, showing that kinematic context alone tracks the full dynamics+kinematics context much more closely than dynamics context alone. They then propose a Volumetric Morphology Encoder (VME): each robot limb is represented as a 3D voxel grid encoding the target limb, the rest of the body, and the local joint region; a 3D convolutional autoencoder compresses each grid into a shared low-dimensional latent token. These tokens replace the body-shape/connectivity portion of the kinematic context fed to ModuMorph/MetaMorph, while the controller architecture and PPO training remain unchanged. Experiments on UNIMAL and a synthetic Mesh-UNIMAL benchmark show that VME often improves or matches baselines, with larger gains on harder tasks and improved transfer to unseen variants/robots, though some regressions and transfer failures remain.

Critical strengths

1.A useful and clearly presented empirical insight.

The context ablation in Section 4 provides a concrete signal that kinematic context, especially body shape/connectivity, is a critical driver of universal-control performance, particularly as task difficulty increases. This is consequential because it motivates where representation effort should be spent, and it makes the paper more than a pure architecture tweak.

2.A reasonable, plug-and-play design with broad compatibility.

VME avoids the primitive-shape assumption and semantic limb labels by learning from voxelized 3D shapes, and it can in principle ingest arbitrary meshes. The experiments include multiple tasks, context ablations (VME*, MatME), transfer settings, and a Mesh-UNIMAL variant. This design and validation make the paper technically coherent and empirically supported, even if the contribution is ultimately incremental.

Critical weaknesses

1.The core contribution is too incremental. The main claim ''body shape should be encoded in its native 3D form rather than as 1D descriptors'' is reasonable, but the proposed method is essentially a standard 3D convolutional autoencoder used to produce context tokens, plugged into existing universal-control architectures without changing the controller or training procedure. The motivating finding that kinematic context matters is interesting but not sufficiently novel or developed into a new problem formulation, algorithm, or theoretical insight.

2.The empirical evidence is not strong or broad enough to establish general significance.

Validation is confined to UNIMAL and a synthetic Mesh-UNIMAL benchmark in simulation, with limited baselines and no real-world or large-scale offline cross-embodiment evaluation. Gains are often meaningful but not uniformly robust: there is a slight MetaMorph degradation on OB, policies still struggle on R_{test} , and VME sometimes underperforms VME* in isolated cases. Mesh-UNIMAL is costly and only evaluated on three tasks. These limitations matter because the paper’s central claim is about the value of 3D morphology representations for universal control broadly, not just for this benchmark suite.

Questions for authors
  1. How much of VME’s gain over MatME is attributable to genuinely using 3D shape information, rather than to differences in autoencoder capacity, pretraining budget, or hyperparameter tuning? A controlled comparison matching parameter count and pretraining compute would substantially clarify the source of the improvement.

2.Are the reported improvements statistically reliable across seeds and tasks, especially where VME only matches or slightly degrades baselines (e.g., MetaMorph on OB)? Please report confidence intervals or paired seed comparisons for the main deltas, and explain the R_{test} transfer failures.

3.Since VME only replaces the body-shape/connectivity portion of the kinematic context, can the authors quantify how much room remains from improving the rest of the context? In particular, would VME still provide gains when combined with stronger dynamics or joint/actuation representations, or is the observed benefit largely specific to the current ModuMorph/MetaMorph context format?

2 comments

Thank you for your detailed review. We appreciate your assessment of the work as "technically coherent and empirically supported", as reflected in your Soundness (3) and Presentation (3) scores. We welcome the opportunity to clarify the points that you raise and address any remaining concerns, and hopefully strengthen your overall assessment of the paper. We have also updated the manuscript (changes in blue) to reflect these clarifications.

W1: Incremental. Our contribution is a finding about the context used in online UC, not a new model architecture. Most prior online UC work (SMP/NerveNet through ModuMorph) upon which we build feeds primitive-shape descriptors as body-shape context. In contrast, we are the first to include and study learned 3D-aware context vectors for online UC controllers. We show that kinematics (particularly body shape) account for much of the context's value (Table 1). Changing only this part of the context yields reliable gains. Across all 20 UNIMAL tasks (Figs. 4, 14), VME beats D+K in 19 out of 20 tasks with ModuMorph (median +14.9%, p<0.001) and 17 out of 20 with MetaMorph (median +10.4%, p<0.001), and gains are also higher in harder tasks. All of this is achieved with the simple, relatively inexpensive context-vector change that we propose.

W2: Breadth. UNIMAL is the only large-|M| online UC benchmark and the only one used by recent work (MetaMorph, ModuMorph, "Distilling Morphology" (Xiong, 2024)). Beyond this benchmark, we reintroduced MVT and introduced Mesh-UNIMAL. We also evaluated ModularRL (Appendix B.3), pinned-velocity and random-rotations variants. Finally, we evaluated prior work's context tokens (D+K), two state-of-the-art controllers, MatME, and ablations (VME*, D, K and ∅ controls).

  • Other benchmarks: no other established large-|M| online UC benchmarks exist. Manipulation benchmarks cover only a small number of gripper finger counts. The closest is URMA (Bohlinger (2024)), but |M| is small and |R|=20 versus UNIMAL's |R|≈2500.
  • Real-world experiments are interesting, but given the very low performance on R_test (and even R_unseen) across all baselines and ablations, in-simulation transfer remains insufficiently solved by this line of work.
  • Offline cross-embodiment experiments are also interesting, but address a different setting. Online UC aims for zero-shot transfer to unseen morphological graphs. In contrast, offline cross-embodiment methods (VLAs, diffusion policies) typically assume known morphologies and do not admit unseen morphological graphs. These are related but distinct subfields.
  • In summary, we studied all existing benchmarks that were reasonably applicable and feasible. Our novel settings bring our evaluation to ≈4 times the scenarios used by recent work in this subfield ((MetaMorph, ModuMorph, "Distilling Morphology" (Xiong, 2024), Bohlinger (2024), etc.).

As for performance:

  • MetaMorph on OB (−1.5%): this is the only negative delta among the 20 in Fig. 4, and not significant (95% CI [−3.1, +2.0] points, p=0.65). In any case, MetaMorph tends to have difficulty to adequately leverage its context. This is what motivated ModuMorph, so these results align with the story told by our study's findings: our contribution is explicitly better context.
  • VME vs. VME*: statistically indistinguishable (VME higher in 7/10 Fig. 4, p=0.44). VME* differs only in providing limb voxels to the encoder. This experiment demonstrates the value of body and joint voxels. VME* also tends to outperform non-3D-aware baselines, further supporting the importance of 3D-aware context for online UC.

Q1: Capacity vs. 3D. Both VME and MatME are trained on R_train only, with L=16. VME has 14.9M params and was trained for 5k epochs (≈60 h), while MatME has 0.65M params and was trained for 30k epochs (≈19 h). Matching parameter counts is undesirable: VME has an input size of 32x32x32=32768 (param/input ratio 455), while MatME's input size is 272 (ratio 2405). MatME is therefore better at reconstructing R_train, R_unseen and R_test. Increasing its parameter count risks overfitting R_train; we are confident we gave MatME its best shot. Even if MatME's offers better reconstruction quality on all three datasets, VME still outperforms it in every case, highlighting the importance of 3D data for this problem.

Robot setVME: CEVME: limbs perfect (same class)MatME: MSEMatME: limbs perfect (allowing 5% absolute error)
R_train0.0850 / 1,170 (0%)5e-51,170 / 1,170 (100%)
R_unseen0.08911 / 28,080 (0.04%)7e-526,474 / 28,080 (94.3%)
R_test0.6670 / 1,117 (0%)3e-2132 / 1,117 (11.8%)

Q2: Statistical analysis. We have added a Statistical Analysis section to the paper. We test aggregate gains across tasks rather than individual tasks: per-task tests would over-emphasize task-specific outcomes, while aggregating across tasks better reflects the method's generality and leverages more statistical power from a higher number of seeds.

  • Training: for each run, we take the mean success over the last 1% of training (the red indicators in Figs. 4 and 14). We report median gains across tasks with stratified bootstrap CIs.
  • Significance: for both training and transfer, we use a van Elteren test, which compares runs within each task and combines the tasks. For transfer, because all evaluation variants of a task share the same trained policy/seed, the test permutes policies/seeds so that they are not treated as independent.
Training (Fig. 4)VME higherMedian gain [95% CI]p
VME vs D+K (ModuMorph)10/10+19.6% [11.9, 28.7]<0.001
VME vs D+K (MetaMorph)9/10+16.9% [2.3, 30.5]0.008
VME vs MatME (ModuMorph)10/10+76.7% [36.4, 125.3]<0.001
VME vs VME* (ModuMorph)7/10+5.6% [−1.8, 13.0]0.44
Transfer (Tab. 2, VME vs D+K)R_unseenR_test
Unmodified27/30, p<0.0014/5, p=0.018
Pinned high velocity24/24, p<0.0014/4, p=0.003
θ30/30, p<0.0015/5, p=0.056
Pinned high velocity + θ24/24, p<0.0013/4, p=0.114
All tasks116/120, p<0.00117/20, p<0.001

Over all 20 tasks of Figs. 4 and 14, VME beats D+K in 19 out of 20 with ModuMorph (median +14.9%, p<0.001) and 17 out of 20 with MetaMorph (median +10.4%, p<0.001), and beats MatME in 20 out of 20 (median +43.1%, p<0.001). With ModuMorph, the gain grows with task difficulty (median +8.3% on unmodified tasks vs. +22.2% on pinned high-velocity + θ tasks). MetaMorph benefits less (P(VME>D+K) = 0.66 vs. 0.79 for ModuMorph). But this is expected: MetaMorph only concatenates the context to each limb's observation, while ModuMorph's hypernetwork and morphology-aware attention were designed specifically to better exploit context.

R_test: Indeed, the data show that R_test is a very difficult problem setting. The morphologies are out of distribution. Intuitively, controlling a body you have never seen before is an incredibly difficult task. Still, our contribution is higher in 17/20 tasks, with P(VME>D+K)=0.69 [0.61, 0.75] and p<0.001 and it is never significantly worse. Even if this regime is very difficult, the fact remains that VME does provide a sizeable improvement, with only a simple change in context vector, further supporting our claim that there are a lot of gains to be harvested from 3D-native morphology context. Overall, the low performance on R_test underscores that this subfield has yet to solve the morphological graph transfer problem, even in simulation. Therefore, hardware experiments would be risky and potentially unsafe.

Q3: Dynamics Context. Overall, this is an interesting question and it is worth a dedicated paper/study. What we have found about dynamics context in this manuscript is the following: in Tab. 1, dropping all dynamics context changes performance by only −1.8% to +2.9%. This is within noise; thus dynamics contexts does not seem to be very important within current large-|M| online UC architecture. In contrast, dropping K costs up to −14.3% performance, and dropping all context costs −17% to −84%. By replacing K (full kinematics context) with VME (learned body shape context only, an imperfect reconstruction of a subsection of K) leads to gains of up to ~35% on that same task (Fig. 11). VME replaces only body shape, so it is complementary to richer dynamics features. Additionally, it already helps when the dynamics change: on the four dynamics variants of R_unseen, VME beats D+K in 77 out of 80 cases (+25%, p<0.001).

We would like to thank you again for your detailed review and for the time and consideration you have devoted to this rebuttal process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.