acceptodds
Under review as a conference paper at ICLR 2027

Representation or Objective? Synergy Action Spaces for Human-Like Dexterous Control

Abstract

Learning dexterous motor control over redundant musculoskeletal systems promises biologically grounded behavior. Still, reinforcement learning tends to discover non-human, high-effort activations. On a dexterous manipulation task, standard policy search learns to complete the task by de-humanizing its muscle use, roughly doubling effort and drifting off the human activation manifold as performance rises. A natural remedy is to constrain where the policy may act by confining it to a low-dimensional coordination manifold, a muscle-synergy action space. We ask whether this suffices to recover human-like dexterous control, and whether the source of the manifold matters, comparing synergies derived from human motion capture, system dynamics, and learned predictive and generative models. We find that a coordination bottleneck restores human-like muscle use in aggregate, and that different sources buy different strengths in task reliability, zero-shot transfer, or coordination, but no action-space prior recovers human-like control at the level of individual movements. Constraining where actions may live does not determine which the policy picks. What does is the objective: a behavioral imitation reward recovers per-movement human-likeness even with no coordination prior at all. We conclude that human-like dexterous motor control is a property of the objective, not the action space, and release the benchmark, synergy bases, and human-data pipeline.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Dextrous Manipulation, Non-Linear Control, RL

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_tW2bOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper compares human-derived, dynamics-derived, and learned action representations on a 78-muscle bimanual typing platform, and adds a per-key template-imitation reward. Separating the roles of structural priors and objectives in human-like control is scientifically interesting. The present evidence, however, supports complementary effects of representations and objectives more than the central claim that human-likeness arises from the objective rather than from the action representation.

Critical strengths
  1. A valuable fine-motor-control platform. Section 3 combines a 78-muscle bimanual model, a contact-capable keyboard, and GPU-parallel training, providing an interesting testbed for detailed dexterous-control questions.

  2. Broad coverage of action representations. Tables 1 and 5 compare linear, predictive, and generative representations and also report approaches that did not deliver the expected gains. This is useful empirical coverage for action-representation research.

  3. Useful tests beyond aggregate success. Sections 4.3-4.4 study keyboard changes and individualized templates, raising transfer and individual-style questions that are relevant in addition to success rate.

Critical weaknesses
  1. The construction and evaluation of human-likeness are not independent. The minimum-effort inverse-dynamics activations in Section 3.3 generate the action prior, the imitation reward, and the human-likeness metric. Better template matching therefore does not establish recovery of real human muscle control without independent behavioral or physiological validation.

  2. The central claim is stronger than the reported comparisons support. Without imitation reward, the human and JoSE spaces already raise m@press from 0.07 to 0.60 and 0.61, while raw plus reward reaches only 0.53. With reward, human and JEPA reach 0.64 and 0.65 in Table 5. These results do not support a simple conclusion that the objective, rather than the representation, is the primary source of human-likeness.

  3. JoSE is not a complete no-human-data condition as claimed. Appendix B states that the latent scales for all synergy directions come from human per-key templates, and the dynamics basis that lacks an embedded mean still uses a human-template mean. An ablation without the human mean and scales is needed.

  4. The main training comparisons are not replicated. Section 4.1 and Appendix A use only seed 0 for all principal conditions, so apparent differences among representation sources, individualization failures, and reward weights cannot be separated reliably from training stochasticity.

Questions for authors
  1. Can the authors evaluate the reward-driven improvement using human data not involved in template construction, with independent kinematic, contact-timing, or feasible EMG-based measures?

  2. Can the conclusion be reformulated in terms of sufficient and complementary effects of action priors and objectives, and can the apparent contradiction between the above-0.60 results in Table 5 and the statement that all conditions remain below that limit be resolved?

  3. After replacing the human-derived mean and latent scales with quantities independent of the human data, does JoSE still support the claim that it does not require human data?

  4. Can the key representation-by-reward conditions be repeated across multiple random seeds, with paired effects and uncertainty intervals?

  5. Can the paper report how often coordinate-wise clipping is active and the residual of activations to the original affine synergy space, so that the practical validity of the low-dimensional-subspace interpretation can be assessed?

1 comment

I keep my final decision.

Reviewer_ZYAtOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper compares action representations and a human-template reward in a bimanual musculoskeletal keyboard task. It argues that the reward, rather than the action representation, determines how human-like the learned control is.

Critical strengths
  1. The benchmark enables useful comparisons of representation and reward choices in a shared bimanual task. Adapting the musculoskeletal model, processing existing human motion data, and running 17 training configurations represent substantial engineering and experimental effort.
Critical weaknesses
  1. The central claim that human-like control depends on the objective rather than the action space is not established by the comparisons. Raw+reward has the lowest PR (4.40), but JoSE without imitation reward has higher m@press (0.61 versus 0.53) at similar press occupancy (Table 5). Lower PR indicates fewer effective muscle dimensions; it does not by itself establish closer human matching. The abstract also claims recovery of per-movement human-likeness without a clear criterion, while Section 4.5 describes performance as approaching the reference. The results support distinct benefits from reward and representation, but Section 5 does not explain how they justify its stronger conclusion.

  2. Section 5’s claim that human-like control does not require human data is not supported by the JoSE experiment. Section 3.2 establishes this independence only for the basis directions. Appendix B and the supplied decoder use human templates to set activation offsets and scales, so the evaluated controller still incorporates human data. The paper extends a claim about basis construction to the full controller without a matched comparison that removes this dependence.

  3. The behavioral interpretation of the m@all–m@press gap in Section 3.4 lacks supporting evidence. Beyond sampling different time points, the supplied evaluator excludes frames without valid human templates from m@all but counts pressed frames of this kind as zero in m@press. The paper does not report whether or how often such frames occur in its evaluation. That information is necessary to determine whether template coverage contributes to the gap attributed to behavior. Figures 3 and 5 provide scores and press success, but no motion or muscle-activation comparisons that substantiate the proposed behavioral explanation.

  4. The human reference is insufficiently defined for the paper’s central comparison. Appendix E describes a recovered human template evaluated in simulation, but all five values in Table 5 match the learned human-PCA policy at the reported precision. The description does not establish how evaluating this reference differs from evaluating that policy. Consequently, the claim of approaching the human reference lacks a clearly specified comparison target.

Questions for authors
  1. What evidence supports attributing human-like control to the objective rather than the action space? Please explain how the PR and m@press results support that claim, and specify whether “recovery” means improved matching or reaching the human reference.

  2. What evidence shows that the JoSE controller retains its reported behavior without human-derived offsets and scales? If no matched comparison is available, the human-data-free claim in Section 5 should be limited to the basis directions.

  3. Please state the template-validity rules used for the reported metrics and the fraction of pressed frames without valid templates. If such frames were included, does the m@all–m@press gap still support the same behavioral interpretation when both metrics exclude them?

  4. What differences in motion and muscle activation account for the reported score differences? Please provide comparisons across methods for the same target keys, alongside the recovered human references, using representative simulated successes and failures.

  5. How does “human (reference)” in Table 5 differ from the learned human-PCA policy? Please specify how it was generated and evaluated, and explain why all five reported values match.

Reviewer_jKhjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the performance of different types of synergy action spaces for musculoskeletal control in terms of task performance and human-likeness. To this end, the paper develops a keyboard-typing benchmark based on the MuJoCo simulator, where a bimanual MyoHand model is controlled by policies trained using Brax-PPO. Using this benchmark, the paper compares raw muscle control, PCA-based action spaces, dynamics-derived action spaces, and learned action spaces, and discusses several findings from these comparisons.

Critical strengths

The paper provides a useful benchmark for studying musculoskeletal control in manipulation tasks.

The comparisons cover a range of representative action spaces, which provide useful information for future work.

Critical weaknesses

The paper does not introduce new control techniques. Its main contribution is to benchmark existing approaches on a rather specific task. The findings are largely intuitive and offer limited new insight. Also, considering that keyboard typing covers a relatively narrow range of manipulation behaviors, it is unclear whether the findings generalize to more complex dexterous manipulation tasks. The technical contribution is thus limited.

The paper is overly condensed, and the writing is dry and sometimes difficult to follow. Key technical details, e.g. the definition of metrics (Sec 3.4) and what are human references in Table 1/2, are too brief to make the paper sufficiently clear and self-contained. Also I find some statement are hard to interpret, e.g. Line 311.

Human-likeness is evaluated against muscle activations estimated from human motion-capture data. If I understand correctly, the same dataset is used to compute the PCA-based, dynamics-derived, and learned action spaces. This may partly explain the high human-likeness scores of some methods in Table 1 and weakens the findings for generalization to unseen human behavior.

Questions for authors

in table 1/2, why does the human reference have lower human-likeness scores than some learned policies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.