- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes an unsupervised skill discovery method that learns skills for controlling specific state components by representing skill critics as linear combinations of basis critics. It also introduces an orthogonality loss to stabilize critic training. To evaluate the method, the authors introduce metrics that characterize the relationship between skills and state components. The results provide evidence of improved controllability on benchmark tasks compared with baselines. The authors also evaluate the method on a simulated Franka robot, demonstrating partial controllability.
Critical strengths
- Building on the observation from prior work that distinguishable skills are not necessarily controllable, the paper proposes a method to address this gap and empirically evaluates its effectiveness through analysis of the learned policies.
- The metrics for evaluating the proposed algorithm (Section 4) are well designed to clearly capture what they aim to assess and are reasonable.
Critical weaknesses
-
Unclear motivation
- As a reviewer and researcher in this field, I understand unsupervised skill discovery as the discovery of meaningful skills through autonomous exploration without external rewards. Here, “meaningful” may refer to skills that are difficult to achieve in terms of temporal distance (METRA [1]), skills that are diverse in a linguistic context (LGSD [2]), or skills that maximize mutual information (DIAYN [3]). However, it is difficult to see the significance of discovering skills that independently control individual joint angles, as proposed in this paper. If independently controlling individual joints is itself the objective, what additional benefit does learning skills provide over controlling the joints directly using a PD controller or direct action commands?
-
Difficulty following the paper
- Terms and expressions that admit multiple interpretations are introduced without sufficient context or concrete examples. Examples include “direct control interface,” “non-target descriptors,” “platform-selected,” and “geometric descriptor.” This makes it difficult for readers to readily understand their intended meaning.
- Figure 2 is particularly difficult to interpret. The legend overlaps with the schematic, and the annotations are densely packed. It is unclear what the arrows and lines refer to, making the underlying geometric intuition difficult to grasp.
-
Concerns regarding the experiments and interpretation of the results
- The fact that the Q-function is linear in the skill variable does not, by itself, imply that joint angles will vary smoothly, that other joints will remain fixed, or that multiple joint targets can be achieved simultaneously. Orthogonality regularization does not guarantee these behaviors either. The paper lacks a theoretical or intuitive explanation of why these effects arise and under what conditions they can be expected.
- The experimental settings are overly simple. Evaluating HalfCheetah and Humanoid in fixed poses and discovering skills that merely correspond to independent joint control are substantially different from the complex, temporally extended behaviors generally studied in the field of unsupervised skill discovery. These results therefore provide only limited evidence that controllable skills can be learned under more general robot dynamics.
At this stage, I lean toward rejection for the reasons outlined in the weaknesses above and the questions in the following section.
[1] Park, Seohong, et al. "METRA: Scalable unsupervised rl with metric-aware abstraction." 2024.
[2] Rho, Seungeun, et al. "Language guided skill discovery." 2025.
[3] Eysenbach, Benjamin, et al. "Diversity is all you need: Learning skills without a reward function." 2018.
Questions for authors
- What additional benefit do the learned skills provide over a PD controller or direct action commands for independent joint control?
- Why should a critic that is linear in the skill variable, together with orthogonality regularization, lead to smooth joint control, limited changes in non-target joints, and simultaneous achievement of multiple joint targets?
- Do the learned skills remain controllable during dynamic, temporally extended behaviors beyond the static-pose settings evaluated in the paper?