acceptodds
Withdrawn from ICLR 2027

DexPrior: Bootstrapping Online Dexterous Adaptation with Multi-Task Latent Priors

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Dexterous manipulation is challenging to learn from scratch due to its contact-rich interactions and long-horizon dynamics. While demonstrations can provide a useful initialization for online adaptation, conventional behavioral policies are typically task-specific and may degrade when trained on heterogeneous multi-task data. We present DexPrior, a framework that learns task-agnostic latent skills from multi-task demonstrations, providing a reusable prior for online adaptation. DexPrior adopts an Encoder–Prior–Decoder architecture to learn reusable skills from shared state representations and privileged information. The pretrained skill prior is then reused for online adaptation, where a diffusion-based action decoder and an additional latent-skill transition enable skill-level exploration within the two-layer MDP formulation. On DexJoCo, we evaluate online adaptation against from-scratch and pretrained-policy RL baselines, demonstrating the effectiveness of multi-task skills for long-horizon dexterous manipulation.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, Dexterous Manipulation, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zmXKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces DexPrior for multitask dexterous manipulation.

Critical strengths

The main idea is reasonable. It is logically beneficial if the model can differentiate between different skills or modes of action for downstream action generation, especially in a multitask setting. However, it's not indeed a novel idea to do so, especially in imitation learning for robotic manipulation.

Critical weaknesses
  1. Real-robot experiments are not included in the evaluation, leaving the method’s effectiveness in the real world unsupported by evidence. This is a critical weakness for a paper on robotic manipulation.
  2. As I understand it, the latent variable z, which conditions action generation, is meant to account for different skills or action modes. However, no visualization of this is provided in the paper. Do similar skills or actions have similar latent representations?
  3. Following up on point 2, the latent ablation in Sec. 4.2 does not seem meaningful or reasonable to me. Since all variants use the same checkpoint, trained end-to-end with the original latent inputs, randomly shuffling these inputs is almost certain to degrade performance. However, this does not, by itself, demonstrate the effectiveness of the latent representation.
  4. The presentation is not very satisfactory. In particular, the analysis of the experimental results is insufficient.
Questions for authors

Is this method scalable? In the paper, 6-tasks mulitask training setting is tested. However, if we want to scale the training to hundreds of tasks in real life, would the latent still be a plus or a drawback?

Reviewer_VMgROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DexPrior. A method proposed for overcoming difficulties faced when learning dexterous manipulation (contacts and long horizon), in particular in the case of multi-task policies where heterogeneity between tasks and multimodal behaviours make it hard to learn solely from demonstration. DexPrior is built on top of DPPO, reusing the two-layer MDP architecture, but complementing it with an “Encoder–Prior–Decoder” architecture. In this setup, the encoder-decoder is first learned offline from a dataset of demonstrations and privileged information to learn a task-agnostic latent skill prior. Then, for online adaptation, the prior is frozen and an additional residual network is trained together with the decoder. The method is evaluated on a set of 6 hand manipulation tasks on DexJoco against both from scratch and pretrained RL baselines.

Critical strengths
  • The motivation is clear and well-argued: BC policies degrade on heterogeneous multi-task data, and a shared latent skill space is a natural answer.
  • Reusing a pretrained prior at the skill level for online adaptation is a more meaningful use of the demonstrations than plain policy initialization.
  • The KL in offline training and then freezing the prior network is a sensible design leaving only lightweight components to be adapted online.
Critical weaknesses
  • Reported gains over baselines are small, and no statistical evidence is given that they are meaningful.
  • There is no comparison with single-task policies or single-task skill priors.
  • Several design choices lack justification or implementation details, e.g. the effect of the residual network, the demonstration data (replay, composition), and the choice of privileged information. In addition, the decoder-backbone and latent-dimension ablations do not fully support presenting the chosen configuration as optimal since the chosen setting is not always best on all metrics.
  • Limitations are incomplete, for example one main limitation is that the method needs an action-labelled demonstration dataset covering the target tasks, which excludes observation-only data and limits scalability.
Questions for authors

Clarifications:

  • Are the differences stated in table 1 statistically significant? You might want to increase the number of seeds to 5 for the results to be more meaningful
  • How does single-tasks policies compare, in sample efficiency and final performance?
  • What is the individual effect of the residual network? Same for the latent smoothness regularization and the horizon of the privileged information?
  • How are the human demonstrations replayed in simulation? And what is the per-task dataset composition?
  • Correlation between offline and online evaluation: Is Table 2 equivalent to iteration 0 of Table 1 and Fig. 3? If yes and since DPPO and DSRL share the same offline start, how do you explain for example that the success rate is higher in Table 2 than at iteration 10 for DSRL in click mouse and pinch tong in Fig. 3?

Suggestions:

  • Could the encoder-prior-decoder be trained without action labels, and what would break?
  • Since the skills are called task-agnostic, do they transfer to a task absent from the demonstrations?

Typos:

  • 4.2 MODEL ANALYSIS Decoder backbone. “and ong-horizon task execution” missing “l”
  • Text citation for PPO is wrong (links to DPPO)
Reviewer_ByY9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

This paper attempts to improve dextrous manipulation, by learning general task-agnostic skills from multi-task demonstrations. This is done through an encoder-decoder structure, where the latent encoding that the decoder conditions on for action generation can be subsequently steered through residual finetuning to adapt to a particular deployment task of interest.

Critical strengths

The strength of this work is its clear and intuitive idea, which is expressed clearly. The overall idea makes sense and is worth trying.

Critical weaknesses

In terms of results, it appears that performance is not consistently better than alternatives, such as in Figure 3, which should be explained. In fact, it seems like in half of the tasks listed DexPrior performs worse than existing simpler and direct approaches.

In terms of robotic behavior, having more online evaluations across suites of tasks, embodiments, and environments is important to establish the approach as general. Even if only in simulation (real-robot experiments would make this more convincing), having only one suite such as DexJoCo is still quite limited in evaluation.

The figures do not have clear captions, would be good to elaborate more on the findings and takeaways of the curves/charts just by looking at it. Currently the captions are simply one-sentence throw-away lines. Also, it is really strange to have entire columns of 0's for Table 1 for PPO and SAC - either suggesting that the comparison is not directly appropriate (such as not utilizing offline information) or an incorrect implementation. Leaning towards the first explanation, these from-scratch results may not be particularly interesting to include because DPPO does use PPO with offline info and DSRL does use SAC with offline info with greater success than from-scratch PPO or SAC - and DexPrior also only works with offline info.

Additional recent approach for the offline skill learning step include DEPS [1]. This would be an interesting alternative for DexPrior, where rather than learning a residual to change the latent, perhaps steering can be done by adjusting its continuous parameterization.

[1] Gupta et al., Learning Parameterized Skills from Demonstrations, NeurIPS 2025.

Questions for authors

Would the authors also be willing to compare against residual policy finetuning [1]? This would be an interesting comparison given that DexPrior also uses residual policies (though internally to steer the latent skill). Positive comparisons would highlight that residual policy steering is more helpful in a learned latent space rather than direct action outputs.

Why did the authors choose to add RND exploration bonuses to PPO and SAC from scratch - was it an important design decision (which would be strange, as all PPO and SAC results were 0)? And were such exploration bonuses added to PPO and SAC when used through DPPO and DSRL, respectively? Did the authors consider experiments that add in exploration bonuses during the finetuning stage of DSRL/DPPO/DexPrior to improve finetuning performance, such as through DF-ExpEnse [2]?

[1] Ankile et al., From Imitation to Refinement - Residual RL for Precise Assembly, ICRA 2025.

[2] Luo et al., DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning, ICML 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.