acceptodds
Under review as a conference paper at ICLR 2027

EigenDEXplore: Structured Exploration for Dexterous Manipulation with Human Priors

Abstract

Dexterous manipulation poses a challenging high-dimensional optimization problem, as useful behaviors require coordinated motion across many hand joints. In reinforcement learning (RL) and sampling-based trajectory optimization, exploration commonly relies on independent robot joint perturbations, making coordinated behaviors difficult to discover. Prior work reduces this search space for grasp learning using low-dimensional spaces of coordinated joint motions learned from human hand data, but this restricts the expressivity required for general manipulation. Some combine learned and joint-space actions to restore expressivity, but this increases dimensionality and introduces redundancy. We study these effects across diverse manipulation settings, varying action dimensionality, exploration strategy, and the source of human data. Our experiments suggest that human-motion priors are most effective when used to structure exploration rather than change the action representation. Motivated by this finding, we propose EigenDEXplore, which induces correlated exploration by adding perturbations along human-derived eigenvectors to independent joint-space noise, leaving the action space unchanged. Across multiple dexterous hands, EigenDEXplore consistently outperforms joint-space and learned action-space baselines in grasping, in-hand reorientation, and contact-rich manipulation. These gains span unstructured and reference-guided RL, trajectory optimization, and sim-to-real deployment, and are largest in settings with less reward shaping and curriculum design.

Primary area:
applications to robotics, autonomy, planning
Keywords:
dexterous manipulation, learning from human videos, reinforcement learning, trajectory optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kzbyOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how human hand-motion priors can improve exploration for dexterous manipulation. It compares direct joint-space control, reduced PCA action spaces, and joint-plus-eigen action representations to investigate whether gains arise from action parameterization or coordinated exploration. Based on these comparisons, it proposes EigenDEXplore, which combines independent joint noise with perturbations along human-derived PCA directions while retaining full joint-space actions. The eigenbasis is fixed during training and is unnecessary for deterministic policy deployment.

The main contribution is a simple exploration mechanism and an empirical study suggesting that human coordination priors are more useful for structuring exploration than constraining action representations. Validation spans reinforcement learning and sampling-based trajectory optimization across four frameworks and six robot hands, with hardware experiments for tool manipulation and in-hand reorientation. Ablations examine source datasets, retained PCA dimensionality, and human-derived versus random directions. Improvements are generally largest for higher-dimensional hands and settings with less reward shaping or curriculum guidance.

Critical strengths
  1. A useful distinction between action representation and exploration. The paper shows that human-derived coordination patterns can improve learning without restricting the policy to a reduced action space or adding redundant action coordinates. EigenDEXplore preserves independent joint control while encouraging coordinated exploration. This provides a clear, practical insight into how to incorporate human priors, supported by comparisons against joint-space, eigen-space, and eigen-residual formulations.

  2. Broad validation of a simple intervention. The evaluation spans reinforcement learning, trajectory optimization, six hand embodiments, and two real-world manipulation settings. The larger benefits on higher-dimensional hands and under reduced training guidance are consistent with the proposed exploration mechanism. This breadth strengthens the case that the method addresses a recurring difficulty in dexterous manipulation and can be useful beyond a single benchmark.

Critical weaknesses

The RL comparisons do not fully disentangle action parameterization from exploration noise. Crossing two expressive action representations (joint and joint-plus-eigen) with three noise choices (joint, eigen, and joint-plus-eigen) yields six combinations, but the paper evaluates only three of these; its fourth baseline uses eigen-only actions. The missing combinations would help isolate the effects of action representation, noise structure, and their interaction. Although trajectory-optimization ablations partially address this gap, the RL results establish the effectiveness of the proposed combination more clearly than they explain the contributions of its individual design choices.

Questions for authors

Could you clarify the choice of RL comparisons and, if available, share results for the remaining combinations of the two fully expressive action representations (joint and joint-plus-eigen) and three noise choices (joint, eigen, and mixed)? In particular, joint actions with eigen-only noise and joint-plus-eigen actions with joint-only noise would help clarify the individual contributions of action parameterization and exploration structure.

1 comment
AuthorsOpenReview ↗

Response to Reviewer kzby

Q1) The full action × noise grid in RL

Could you clarify the choice of RL comparisons and, if available, share results for the remaining combinations of the two fully expressive action representations (joint and joint-plus-eigen) and three noise choices (joint, eigen, and mixed)? In particular, joint actions with eigen-only noise and joint-plus-eigen actions with joint-only noise would help clarify the individual contributions of action parameterization and exploration structure.

The paper compares the formulations that prior work uses. Eigen-Space makes the human prior the action space (Agarwal et al., 2023; Lum et al., 2024; Yuan et al., 2025), and Eigen-Residual adds eigen actions to joint actions (Gupta et al., 2026b). The remaining combinations are controls, and we have now trained them on DeXtreme. We follow the protocol of Fig. 4 and report consecutive successes at 2B frames as mean ± std across seeds.

Actions ↓ / Noise →Joint onlyEigen onlyJoint + Eigen
Joint8.8 ± 2.3 (Joint-Space)5.7 ± 1.412.3 ± 2.3 (EigenDEXplore)
Joint + Eigen9.7 ± 1.69.8 ± 1.910.6 ± 1.9 (Eigen-Residual)

Eigen-only noise uses the full set of human directions, one per hand joint, so it still spans the joint space. The new cells use 3 seeds.

With joint actions, exploration structure has a large effect. Mixed noise performs best and eigen noise alone performs worst (5.7). With joint-plus-eigen actions, the three noise choices perform similarly (9.7 to 10.6). Under mixed noise, joint actions outperform joint-plus-eigen actions (12.3 vs. 10.6). The full grid is in the revised paper (App. A.2, Fig. 11).

Reviewer_Se29OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EigenDEXplore, a structured exploration method for dexterous manipulation. The method extracts joint coordination patterns from human motion data using PCA and incorporates them into Gaussian exploration noise, encouraging coordinated movements while preserving full action-space expressiveness. Experiments across multiple dexterous manipulation benchmarks and real-world robotic tasks demonstrate improvements over conventional joint-space exploration, particularly in high-dimensional settings with limited reward shaping.

Critical strengths
  1. Simple and effective structured exploration. The paper introduces an intuitive approach to incorporating human motion priors into exploration without restricting policy expressiveness. The method is easy to implement and compatible with existing RL frameworks.
  2. Comprehensive experimental validation. The experiments cover diverse dexterous hands, manipulation tasks, and optimization frameworks, including real-world evaluations. The consistent improvements support the practical effectiveness of the proposed method.
Critical weaknesses
  1. Limited methodological novelty. The use of PCA-based human motion priors for dexterous manipulation has been explored in prior works, such as DextrAH-G [1] and Dexterous Functional Grasping [2]. The proposed method mainly extends this idea by incorporating PCA directions into Gaussian exploration covariance rather than action representation. While effective, the technical novelty appears relatively incremental compared to existing PCA-based action priors and structured exploration approaches.

  2. Insufficient comparisons with relevant exploration baselines. The experimental comparisons primarily focus on different action representations, with limited evaluation against existing structured exploration methods, such as learned covariance policies. Furthermore, recent generative policy approaches, including DPPO [3], FPO++ [4], and RoboFFT [5], can leverage expressive action distributions and RL fine-tuning to achieve structured exploration. Comparisons with these approaches would help clarify the effectiveness and advantages of PCA-based exploration.

Questions for authors

Q1. How does EigenDEXplore compare with existing structured exploration approaches, such as Gaussian policies with learned full or low-rank covariance matrices? Such comparisons would help establish the specific advantages of human-derived PCA priors.

Q2. Can the authors provide more direct evidence that the performance improvements arise from more effective exploration of coordinated actions, rather than simply introducing correlations into the exploration distribution?

Q3. Instead of extracting human motion priors through PCA, could a diffusion or flow-based policy be pretrained on human demonstrations and subsequently fine-tuned using RL methods such as DPPO [3], FPO++ [4], or RoboFFT [5]? Such approaches may capture more expressive, nonlinear, and state-dependent coordination patterns. Could they achieve comparable or better performance, and what are the specific advantages of the proposed method?

[1] Lum et al., DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabrics.

[2] Agarwal et al., Dexterous Functional Grasping.

[3] Ren et al., Diffusion Policy Policy Optimization (DPPO).

[4] Yi et al., Flow Policy Gradients for Robot Control (FPO++).

[5] Li et al., RoboFFT: Finetuning Generative Robot Policy via Online Reinforcement Learning with Forward Process.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Se29

W1) Novelty vs prior PCA-prior work

The use of PCA-based human motion priors for dexterous manipulation has been explored in prior works, such as DextrAH-G [1] and Dexterous Functional Grasping [2]. The proposed method mainly extends this idea by incorporating PCA directions into Gaussian exploration covariance rather than action representation. While effective, the technical novelty appears relatively incremental compared to existing PCA-based action priors and structured exploration approaches.

Prior work, including DextrAH-G and Dexterous Functional Grasping, incorporates human synergies into low-dimensional action representations to simplify learning and encourage coordinated movements. However, this also restricts expressivity, and the benefits of the action representation versus the exploration it induces have remained entangled.

Our key finding is that the benefit can instead be obtained through structured exploration alone, without restricting or expanding the action representation. This preserves full joint-space expressivity while encouraging coordinated exploration, including for tasks where reduced eigen-space actions struggle (0.8 vs. 12.3 consecutive successes in DeXtreme, Fig. 4).

Because this insight concerns the sampling distribution rather than the policy, it generalizes beyond RL to sampling-based trajectory optimization. In SPIDER, changing only the sampling distribution reduces retargeting cost across all six hands.

This distinction between the role of human priors in exploration versus action representation, supported across RL, trajectory optimization, and real-world deployment, is the central contribution of our work.

Q1) Learned covariance baselines

How does EigenDEXplore compare with existing structured exploration approaches, such as Gaussian policies with learned full or low-rank covariance matrices? Such comparisons would help establish the specific advantages of human-derived PCA priors.

We trained a Gaussian policy with a low-rank covariance of the same form as EigenDEXplore in SimToolReal for 9B frames, with directions learned during training from a random initialization. We also include Lattice (Chiappa et al., 2023), which learns state-dependent correlated noise. We report consecutive successes as mean ± 95% CI across seeds.

MethodConsec. successes
Joint-Space7.9 ± 0.8 (5 seeds)
Lattice8.6 ± 2.8 (3 seeds)
Learned low-rank covariance8.6 ± 1.8 (3 seeds)
EigenDEXplore10.6 ± 0.6 (5 seeds)

Learning the covariance improves over Joint-Space, but the policy must discover useful correlations during training. EigenDEXplore starts from correlations observed in human motion and performs best. The results are in the revised paper (App. B.3, Fig. 15).

Q2) Coordinated exploration vs just correlation

Can the authors provide more direct evidence that the performance improvements arise from more effective exploration of coordinated actions, rather than simply introducing correlations into the exploration distribution?

We ran a control that isolates this. It is identical to EigenDEXplore, with the same low-rank covariance, the same per-direction noise scales and the same training setup, except that the human PCA directions are replaced with random directions. The exploration is therefore equally correlated but not along human coordination patterns. In SimToolReal at 9B frames (mean ± 95% CI across seeds):

MethodConsec. successes
Random directions7.7 ± 1.6 (3 seeds)
Joint-Space7.9 ± 0.8 (5 seeds)
EigenDEXplore10.6 ± 0.6 (5 seeds)

Correlated noise along random directions performs no better than Joint-Space, so the improvement comes from exploring along human coordination patterns and not from correlation alone. This agrees with the random-direction analysis in SPIDER (App. B.2). The results are in the revised paper (App. B.2, Fig. 15).

Q3) Generative policies fine-tuned with RL

Instead of extracting human motion priors through PCA, could a diffusion or flow-based policy be pretrained on human demonstrations and subsequently fine-tuned using RL methods such as DPPO, FPO++, or RoboFFT? Such approaches may capture more expressive, nonlinear, and state-dependent coordination patterns. Could they achieve comparable or better performance, and what are the specific advantages of the proposed method?

Methods such as DPPO fine-tune a policy pretrained on task demonstrations in the robot's action space, which our setting does not have. Our human data contains hand postures only, without the robot, objects, or task. EigenDEXplore needs one PCA of these postures per hand, reused across tasks, and trains from scratch with standard PPO.

FPO++ can also train without pretraining, exploring through its base noise, which can be shaped by the same human covariance. We attempted this in SimToolReal, but we were not able to train a stable policy in time for the rebuttal.

Reviewer_DP2YOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies RL exploration in dexterous manipulation. Since useful manipulation behaviors require coordinated motion across fingers and joints, independent per-joint noise explores inefficiently. On the other hand, restricting the action space to a human-derived PCA subspace limits expressivity, while combining PCA and joint actions introduces redundancy. The authors propose to keep plain joint-space actions but explore with both noise along the PCA directions and independent per-joint noise.

Specifically, EigenDEXplore retargets hand poses from large-scale human egocentric video to each robot hand and extracts principal components to define these exploration directions. The authors first compare four formulations, Joint-Space, Eigen-Space, Eigen-Residual, and EigenDEXplore, and find that restricting or expanding the action space both hurt performance, while structuring exploration alone gives the best results. They then show that EigenDEXplore consistently outperforms Joint-Space across various tasks, with larger gains on higher-DoF hands and under sparser rewards. Ablations further suggest that the benefit comes from human coordination patterns rather than from correlated noise alone.

Critical strengths

Simple and practical method. The proposed method modifies only the exploration distribution, leaving the network, action space, and reward unchanged. This makes it plug-and-play for existing RL and sampling-based optimization pipelines. The extensive experiment results also show the effectiveness of the method.

Well-motivated question supported by thorough experiments. Prior work couples the use of human priors in the action representation with their effect on exploration, making it unclear where the benefits come from. The paper cleanly disentangles these factors through a controlled comparison of four formulations, and the resulting finding is validated across six hands, RL and trajectory optimization, and real-world deployment. The gains follow interpretable trends, growing with hand DoF and under sparser rewards. The ablations on the source dataset and the retained PCA variance further justify the main design choices.

Critical weaknesses

Unclear advantage over Eigen-Residual. The central claim that human priors should structure exploration rather than change the action space rests heavily on the comparison with Eigen-Residual, which shares the same exploration noise and differs only in its redundant PCA action parameterization. However, the results of the two methods are close, with overlapping confidence intervals in Fig. 3 and Fig. 4. Moreover, the paper attributes the gap only to “a cost to expanding the action space” without explaining the underlying mechanism, for example whether it stems from drift along redundant directions or from PPO computing likelihoods and entropy in the higher-dimensional action space. As a result, the experiments show that the redundant PCA action path brings no additional benefit, but it remains unclear whether and why it is actually harmful.

Questions for authors

Comparison with Eigen-Residual. We understand that the extended training runs are computationally expensive, but additional experiments (e.g., new, but less costly settings) and a statistical significance test would help confirm the gap with Eigen-Residual. Also, it could be better if authors can explain the mechanism behind such gap.

Comparison with learned correlated exploration. How does EigenDEXplore compare with methods that learn correlations across actuators without human data, such as Chiappa et al. (2023)?

Evolution of the learned noise. How do the learned noise scales for the independent joint noise and the PCA-direction noise evolve during PPO training? For example, do some principal components collapse while others remain active, and does the prior mainly matter early in training? Such an analysis would give more direct insight into how the human prior shapes exploration.

2 comments
AuthorsOpenReview ↗

Response to Reviewer DP2Y

Q1) Is the gap over Eigen-Residual real?

We understand that the extended training runs are computationally expensive, but additional experiments (e.g., new, but less costly settings) and a statistical significance test would help confirm the gap with Eigen-Residual.

We ran significance tests on the per-seed results. In SimToolReal at 9B frames, EigenDEXplore outperforms Eigen-Residual (10.6 vs. 9.5, Welch's t-test, p = 0.03). In DeXtreme the difference is in the same direction (12.3 vs. 10.6) but is not significant given the variance across seeds (p = 0.29). We will report 10 seeds for all methods in DeXtreme in the camera-ready version.

W1) Why does the redundant action path hurt?

The paper attributes the gap only to "a cost to expanding the action space" without explaining the underlying mechanism, for example whether it stems from drift along redundant directions or from PPO computing likelihoods and entropy in the higher-dimensional action space.

We tested both in SimToolReal. For drift, we rolled out the trained Eigen-Residual policies and measured how much of each action lies along the redundant directions, where the joint and eigen actions cancel and leave the target unchanged. This is 14% of the action, below the 24% obtained when the joint and eigen actions are paired at random, so we find no drift. The second mechanism is present. In Eigen-Residual, the noise on the eigen actions grows to about 1.5× its initial value over 9B frames, while in EigenDEXplore PPO reduces the noise along the human directions to about 0.4× (App. A.3, Fig. 12).

Eigen-Residual adds the eigen actions as additional policy outputs, and PPO's entropy bonus rewards noise on every output. The noise on the eigen actions therefore receives its own reward, and it increases. EigenDEXplore adds no outputs. Its noise along the human directions shares the reward on the joint actions with the independent noise, i.e., the two noises compete and PPO keeps each only where it helps, rather than being rewarded for both. As a result, PPO reduces the noise along the human directions as the policy improves (Q3). We thank the reviewer for the suggestion, and we have revised the paper to describe this mechanism in place of "a cost to expanding the action space".

Q2) Learned correlated exploration without human data

How does EigenDEXplore compare with methods that learn correlations across actuators without human data, such as Chiappa et al. (2023)?

We trained three such methods in SimToolReal for 9B frames. Lattice (Chiappa et al., 2023) learns correlated, state-dependent noise by perturbing the policy's last hidden layer. The other two use a low-rank Gaussian covariance with the same form as EigenDEXplore, with random directions that are either kept static or learned during training. We report consecutive successes as mean ± 95% CI across seeds.

MethodConsec. successes
Static low-rank covariance7.7 ± 1.6 (3 seeds)
Joint-Space7.9 ± 0.8 (5 seeds)
Lattice8.6 ± 2.8 (3 seeds)
Learned low-rank covariance8.6 ± 1.8 (3 seeds)
EigenDEXplore10.6 ± 0.6 (5 seeds)

Learning the correlations improves over Joint-Space, and EigenDEXplore performs best. For Lattice, we follow the official implementation with two changes to train stably in this setting. We normalize the latent that drives the noise, and we use a constant learning rate. Lattice underperforms without these changes, which deviate from the training protocol used in the rest of our experiments. The results are in the revised paper (App. B.3, Fig. 15).

Q3) How the learned noise evolves

How do the learned noise scales for the independent joint noise and the PCA-direction noise evolve during PPO training? For example, do some principal components collapse while others remain active, and does the prior mainly matter early in training?

We logged the learned noise scales throughout training (App. B.4, Fig. 16). The PCA noise and the independent joint noise start at a ratio of 1:1. In SimToolReal, the PCA-direction scales shrink steadily while the joint noise stays roughly constant, and the ratio falls to 0.4:1 by 9B frames. No direction collapses in this period. During the tolerance curriculum of the 60B run, the ratio falls further to 0.07:1, and the directions separate. Most fade into the joint noise, while PC 2 ("Claw") remains several times larger than the others.

The prior therefore shapes exploration mainly early in training, potentially guiding the policy toward strategies that it later refines with mostly independent noise.

Thank you for the detailed response and the additional experiments, which address most of my concerns. One point remains on the comparison with Eigen-Residual: the gap is significant in only one of the two benchmarks, and the entropy-bonus explanation is supported only by correlational evidence. If the entropy bonus is the cause, an Eigen-Residual variant that computes the likelihood and entropy over the resulting joint targets, rather than over the joint and eigen outputs separately, should close the gap, and I would suggest this experiment as a direct test of the mechanism.

Overall, the experimental design is solid and the paper is practically useful, but I find the methodological novelty modest, so I keep my rating at 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the reviewers for their feedback. The revised paper adds the following, each discussed in the replies below.

  • Action and noise combinations. We trained every combination of action representation (joint actions, or joint plus eigen actions) and exploration noise (independent joint noise, noise along the human directions, or both) in DeXtreme. With joint actions the choice of noise has a large effect, while with joint plus eigen actions the three noise choices perform similarly, and joint actions with both noises (EigenDEXplore) perform best (App. A.2, Fig. 11; kzby Q1).
  • Correlated exploration baselines. We compare against two methods that learn correlated exploration noise during training without human data, Lattice (Chiappa et al., 2023) and a Gaussian policy with a learned low-rank covariance, in SimToolReal. EigenDEXplore outperforms both (App. B.3, Fig. 15; DP2Y Q2, Se29 Q1).
  • Random directions. We replace the human directions in EigenDEXplore with random directions and keep everything else the same. This removes the improvement over Joint-Space and potentially hurts, so the gain comes from human coordination patterns rather than from correlated noise alone (App. B.2, Fig. 15; Se29 Q2).
  • Eigen-Residual analysis. We examine why Eigen-Residual, which adds eigen actions to the joint actions, trails EigenDEXplore. PPO's entropy bonus rewards the noise on every policy output, so the noise on Eigen-Residual's added eigen actions grows during training. In EigenDEXplore, the noise along the human directions and the independent joint noise act on the same joint actions and compete, and PPO keeps each only where it helps (App. A.3, Fig. 12; DP2Y W1).
  • Learned noise. We show how the learned noise scales evolve during training. The noise along the human directions shrinks relative to the independent joint noise, so the prior shapes exploration mainly early in training (App. B.4, Fig. 16; DP2Y Q3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.