acceptodds
Withdrawn from ICLR 2027

SelfSkill: Self-State Conditioned Skill Discovery for Embodied Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied agents have achieved remarkable progress in interacting with the physical world, yet existing evaluations primarily focus on task-level outcomes, which reveal whether an agent succeeds but provide little insight into what capabilities an agent possesses. In this work, we introduce SelfSkill, a framework for discovering skills of embodied agents. The key insight is that the skills are not fixed behavioral patterns, but emerge from how an agent utilizes different environmental factors under its current embodied conditions, which we characterize as self-states. To characterize such skills, SelfSkill constructs a initial behavioral basis space, where each basis represents a potential behavioral influence induced by an environmental factor. Given trajectory observations, SelfSkill infers the activation pattern over these bases under different self-states, producing a skill that captures how an agent generates behaviors under varying self-states. By associating discovered skills with trajectory outcomes, SelfSkill reveals the strengths and limitations of different agents across self-states. Furthermore, the discovered skills provide interpretable guidance for agent enhancement by identifying complementary capabilities among different agents and transferring effective behavioral knowledge. Experiments on ObjectGoal Navigation and Open-Vocabulary Mobile Manipulation demonstrate that SelfSkill offers a new perspective on understanding, evaluating, and improving embodied agents.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied Agents, Skill Discovery, Self-State Conditioning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yjNbOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

SelfSkill represents existing embodied agents through self-state-conditioned weights over semantic and geometric movement factors. It uses these representations to analyze behavior and guide navigation, reporting improvements on ObjectNav and OVMM.

Critical strengths

The self-state-conditioned representation offers a useful way to compare agents beyond aggregate success rates.

The intervention gains are consistent across benchmarks

Critical weaknesses

The skill-ranking reward does not credit progress toward eventual success. Nonterminal exploration and recovery fragments receive only a step penalty, so useful behavior can be scored no better than ineffective behavior. This undermines the ranking used to select complementary skills.

The central benefit of self-state conditioning is not established experimentally. The paper shows improvements from the complete intervention, but never compares against the same factor model and expert trajectories with weights shared across self-states. (Section 3.4 introduces state-specific activation patterns; Tables 3–4 test regularization and intervention weights rather than removing this conditioning.)

Questions for authors

How does the reward distinguish successful exploration or recovery from unproductive motion before goal completion?

How much do self-state-specific weights improve prediction and downstream success over a shared weight vector fitted to the same data?

3 comments

We thank the reviewer for raising these important concerns.

(W1) Clarification of the Reward Formulation


SelfSkill discovers interpretable behavioral strategies by constructing a behavioral basis space and inferring the activation patterns of different behavioral factors under various self-states. These learned representations also provide the foundation for evaluating candidate policies in the subsequent skill-aware policy fusion.

We apologize that the original manuscript did not fully describe the reward formulation used in our experiments, which consists of three complementary components: task completion outcome, step penalty, and BasicSkill-guided motion alignment:

R_k^{\mathrm{success}} + R_k^{\mathrm{step}} + \lambda_a R_k^{\mathrm{align}}.$$ The first two components evaluate task completion and execution cost, respectively. More importantly, the alignment component directly connects policy evaluation with the behavioral representations learned by SelfSkill. Specifically, BasicSkill denotes a set of skills derived from optimal paths collected on the training dataset. To construct a reference motion representation, we leverage the behavioral basis instances and the BasicSkill activation vector $\boldsymbol{\alpha}_k^{\mathrm{Basic}}$ learned by our method: $$\hat{\mathbf v}_k^{\mathrm{Basic}}= \mathbf F_k \boldsymbol{\alpha}_k^{\mathrm{Basic}},$$ where $\mathbf F_k$ represents the behavioral basis instances associated with fragment $k$. For a fragment generated by policy $\pi$, we extract its actual motion representation $\mathbf v_k^\pi$. The alignment reward is then defined as: $$R_k^{\mathrm{align}}= \frac{ (\mathbf v_k^\pi)^\top \hat{\mathbf v}_k^{\mathrm{Basic}} }{ \lVert\mathbf v_k^\pi\lVert_2 \lVert\hat{\mathbf v}_k^{\mathrm{Basic}}\lVert_2+\epsilon }.$$ This component measures the consistency between the actual behavioral tendency of a candidate policy and the reference behavioral patterns represented by BasicSkill in the SelfSkill space. Importantly, this establishes a direct connection between skill discovery and policy evaluation: the behavioral basis space and activation representations learned during skill discovery are reused to construct a skill-based reference for evaluating candidate policies. Therefore, the alignment reward is not an independent, manually specified motion heuristic, but an evaluation signal grounded in the behavioral representations learned by SelfSkill. We emphasize that this reward does not introduce an additional optimization objective for the skill activation vector. The original fitting procedure of $\boldsymbol{\alpha}$ remains unchanged. The learned BasicSkill activation vector is only used to construct the reference motion representation. We will explicitly clarify the complete reward formulation and its connection to skill discovery in the revised manuscript. --- ### (Q1) The Reward Distinguish Successful Exploration or Recovery from Unproductive Motion We clarify that nonterminal trajectory fragments do not receive only a step penalty. Fragments belonging to successful episodes receive the shared episode-level success reward, while all fragments are additionally evaluated through their individual alignment rewards. The two signals serve complementary purposes. The success reward provides global task outcome feedback, whereas the alignment reward provides local behavioral feedback by measuring the consistency of each fragment's actual motion with the BasicSkill reference representation. Consequently, even when an exploration or recovery fragment does not directly approach the goal, it can still receive a positive alignment reward if its motion is consistent with the behavioral patterns represented by BasicSkill. In contrast, motion that deviates from these reference patterns receives a lower alignment score.

(W2 & Q2) Self-state-specific Weights vs. a Shared Weight Vector


To address this concern, we introduce a controlled comparison, where both variants use training data, and activation inference objective to get BasicSkill with State-specific Activation and Shared Activation. The only difference is whether activation weights are estimated separately for different self-states or shared across all self-states.

Specifically, the state-specific variant estimates a separate activation vector for each self-state: αS=arg⁡min⁡αLS(α),{\alpha}^{S}=\arg\min_{{\alpha}}\mathcal L_S({\alpha}), where LS\mathcal L_S denotes the original activation inference objective evaluated on trajectory fragments belonging to self-state SS. In contrast, the shared-weight variant pools trajectories from all self-states and estimates a single activation vector:

\arg\min_{{\alpha}} \sum_{S\in\mathcal S} \mathcal L_S({\alpha}).$$ We evaluate both variants from two complementary perspectives: 1. **Behavioral prediction:** We compare motion reconstruction error on held-out trajectory fragments to examine how well each representation explains the observed behavioral differences. 2. **Downstream navigation performance:** We use the resulting activation representations in the same skill-aware policy fusion or intervention procedure, and compare navigation Success Rate (SR) and SPL under identical evaluation settings. **Table 2. Comparison between State-specific and Shared Activation Weights on HM3D-v1. Fusion and Intervention represent the downstream methods: VLFM+ApexNav and ApexNav+BasicSkills, respectively.** | Variant | Reconstruction Error ↓ | (Fusion) SR (%) ↑ | (Fusion) SPL (%) ↑ | (Intervention) SR (%) ↑ | (Intervention) SPL (%) ↑ | |---|---|---:|---:|---:|---:| | Shared Activation | 0.66 | 55.2 | 26.9 | 57.0 | 30.4 | | State-specific Activation | 0.37 | 60.4 | 33.2 | 62.2 | 33.7 | This comparison directly isolates the contribution of self-state-specific activation patterns. Improvements in both behavioral prediction and downstream navigation performance demonstrate that distinguishing embodied conditions provides benefits beyond using a shared set of behavioral weights.

I appreciate the detailed rebuttal, which addressed most of my concerns. However, reading upon rebuttal and concerns from other reviewers. This seems to be a substantial revision to address all weaknesses. So I'll keep my original rating.

Reviewer_y97pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SelfSkill, a framework for discovering and evaluating the underlying skills of existing embodied agents. The key idea is to represent skills as self-state-conditioned activation patterns over interpretable behavioral bases, such as semantic objects, frontiers, open spaces, and boundaries. Given trajectories collected from pretrained agents, SelfSkill infers these activation patterns by combining trajectory reconstruction with LLM-generated semantic priors. The discovered skills are then used to evaluate agents' capabilities under different self-states and improve existing policies through state-dependent policy fusion and skill-guided intervention. Experiments on ObjectGoal Navigation (ObjectNav) and Open-Vocabulary Mobile Manipulation (OVMM) demonstrate that SelfSkill can improve task success rates over existing baseline agents.

Critical strengths
  1. Extensive Experimentation: The paper provides extensive experimental evaluations across both ObjectGoal Navigation (ObjectNav) and Open-Vocabulary Mobile Manipulation (OVMM), covering diverse benchmarks and baseline algorithms.

  2. Plug-and-Play Framework: The proposed method can be applied to existing pretrained policies without modifying their architectures or requiring additional policy training, making it practical and easy to integrate.

  3. Improved Interpretability: The proposed framework provides an interpretable representation of agent behaviors by decomposing them into meaningful behavioral factors, offering insights into how different agents make decisions under different conditions.

Critical weaknesses
  1. Unclear Motivation and Novelty of Self-State-Conditioned Skills

The authors argue that skills should not be viewed as fixed behavioral patterns but should instead be conditioned on self-states. However, I find this motivation unconvincing, as existing skill discovery methods already learn state-conditioned policies, typically formulated as (\pi(a \mid s, z)). Even with a fixed skill latent (z), the policy can produce different actions depending on the current state (s). Therefore, it is unclear why existing skills are characterized as fixed behavioral patterns or what fundamental limitation the proposed self-state conditioning addresses.

Furthermore, the proposed self-state appears to be a hand-designed abstraction of the agent's state, observation history, and task goal. I do not see a clear conceptual distinction between the proposed self-state and the state information already used in conventional RL policies. The authors should clarify how their formulation differs from existing state-conditioned skill representations and what additional capabilities it provides.

  1. Concerns about Skill-aware Policy Fusion (Section 3.5.2)

I have two concerns regarding the proposed Skill-aware Policy Fusion strategy.

2-1. Generalization of State-Conditioned Policy Rewards

How are the state-conditioned policy rewards estimated? Are they computed exclusively from training trajectories, or do they use information from the evaluation environments?

I assume that these rewards are estimated from training trajectories. If so, the proposed approach relies on the assumption that the relative performance of different policies under each self-state remains consistent across training and unseen test environments. However, it is unclear whether this assumption holds. As I pointed out earlier, the proposed self-states are merely hand-engineered, high-level features of the original state and may not capture sufficient information to reliably predict which policy will perform better in an unseen environment. How do the authors justify or validate this assumption?

2-2. Assignment of Task Completion Rewards to Trajectory Fragments

How is the task completion reward assigned to intermediate trajectory fragments? Does every fragment belonging to a successful episode receive the same success reward?

If so, I question whether this provides a reliable estimate of policy performance under each specific self-state. Simply assigning the same episode-level success reward to all fragments does not necessarily indicate that the policy performed well in every self-state encountered during that episode. Consequently, the resulting state-conditioned reward estimates may not accurately reflect the policy's actual capabilities under different self-states.

  1. Overly Complicated Methodology with Multiple Engineering Components

I find the proposed methodology overly complicated, consisting of numerous hand-engineered components, including:

  1. Hand-crafted self-state definitions (e.g., movement, perception, and interaction status).
  2. Manually designed behavioral basis categories (e.g., semantic objects, frontiers, open spaces, and boundaries).
  3. LLM-based semantic clustering of environmental objects.
  4. LLM-generated priors for skill activation inference.
  5. Trajectory segmentation based on manually defined events and thresholds.
  6. Skill activation inference with multiple regularization terms and hyperparameters.
  7. State-dependent policy selection and weighted skill-guided intervention. While each component may be individually reasonable, the overall framework appears to be a combination of many engineering choices rather than a unified and principled approach. Such complexity raises concerns about potential overfitting to the specific training and validation environments, as numerous design choices and hyperparameters may have been tailored to the benchmarks used in this paper.

Furthermore, it is difficult to identify which components are actually responsible for the reported performance improvements. I believe the paper would benefit from systematic ablation studies and comparisons with simpler alternatives to justify the necessity of these components and demonstrate the generalizability of the proposed framework.

Questions for authors

I would appreciate it if the authors could address the concerns raised in the Weaknesses section.

4 comments

(W1) Motivation and Novelty


We thank the reviewer for raising this important question. We agree that existing skill discovery methods can learn state-dependent policies, and our novelty does not lie in introducing state dependence to skills. Instead, our motivation stems from two limitations of existing skill representations:

1. Interpretability. Existing methods often represent skills through latent variables or learned policies, which do not explicitly reveal how different environmental factors contribute to the agent's behaviors. SelfSkill aims to discover interpretable functional skills that characterize how agents utilize environmental factors to handle different situations.

2. Model-agnostic Representation and Cross-model Fusion. Skills learned by different agents are typically embedded in model-specific representations, making direct capability comparison and cross-model fusion difficult. SelfSkill addresses this by mapping heterogeneous agents' behaviors into a shared behavioral basis space, without requiring access to their internal architectures or latent representations. This provides a common representation for identifying complementary capabilities and enabling cross-model skill fusion.

To achieve these goals, we deliberately formulate observed motion tendencies as combinations of interpretable behavioral bases. Each basis represents an interpretable environmental factor, while its activation characterizes how an agent utilizes that factor under a particular self-state. The self-state organizes trajectories according to different embodied conditions, while the vector decomposition makes behavioral contributions explicit and comparable across agents. This is a modeling abstraction for interpretability and cross-model alignment, rather than an assumption that navigation behaviors are intrinsically linear.

(W2) Concerns about Skill-aware Policy Fusion (Section 3.5.2)


We thank the reviewer for raising these important concerns.

Clarification of the Reward Formulation. SelfSkill discovers interpretable behavioral strategies by constructing a behavioral basis space and inferring the activation patterns of different behavioral factors under various self-states. These learned representations also provide the foundation for evaluating candidate policies in the subsequent skill-aware policy fusion.

We apologize that the original manuscript did not fully describe the reward formulation used in our experiments, which consists of three complementary components: task completion reward, step penalty, and BasicSkill-guided motion alignment:

R_k^{\mathrm{success}} + R_k^{\mathrm{step}} + \lambda_a R_k^{\mathrm{align}}.$$ The first two components evaluate task outcomes and execution cost, respectively. More importantly, the alignment component directly connects policy evaluation with the behavioral representations learned by SelfSkill. Specifically, BasicSkill denotes a set of skills derived from optimal paths collected on the training dataset. To construct a reference motion representation, we leverage the behavioral basis instances and the BasicSkill activation vector $\boldsymbol{\alpha}_k^{\mathrm{Basic}}$ learned by our method: $$\hat{\mathbf v}_k^{\mathrm{Basic}}= \mathbf F_k \boldsymbol{\alpha}_k^{\mathrm{Basic}},$$ where $\mathbf F_k$ represents the behavioral basis instances associated with fragment $k$. For a fragment generated by policy $\pi$, we extract its actual motion representation $\mathbf v_k^\pi$. The alignment reward is then defined as: $$R_k^{\mathrm{align}}= \frac{ (\mathbf v_k^\pi)^\top \hat{\mathbf v}_k^{\mathrm{Basic}} }{ \lVert\mathbf v_k^\pi\lVert_2 \lVert\hat{\mathbf v}_k^{\mathrm{Basic}}\lVert_2+\epsilon }.$$ This component measures the consistency between the actual behavioral tendency of a candidate policy and the reference behavioral patterns represented by BasicSkill in the SelfSkill space. Importantly, this establishes a direct connection between skill discovery and policy evaluation: the behavioral basis space and activation representations learned during skill discovery are reused to construct a skill-based reference for evaluating candidate policies. Therefore, the alignment reward is not an independent, manually specified motion heuristic, but an evaluation signal grounded in the behavioral representations learned by SelfSkill. We emphasize that this reward does not introduce an additional optimization objective for the skill activation vector. The original fitting procedure of $\boldsymbol{\alpha}$ remains unchanged. The learned BasicSkill activation vector is only used to construct the reference motion representation. We will explicitly clarify the complete reward formulation and its connection to skill discovery in the revised manuscript.

2-1. Generalization of State-Conditioned Policy Rewards


Cross-Environment Reward Consistency. We believe the generalization of self-state-conditioned policy rewards is supported by the following considerations. First, self-states provide a shared semantic reference for grouping trajectory fragments across different environments. Although scene layouts and object configurations vary, agents can encounter comparable embodied conditions, such as being stuck or navigating without observing the target. This allows policies to be compared under semantically comparable conditions.

Second, our reward evaluates actual behavioral quality through task completion, execution cost, and motion-goal alignment, rather than predicting policy performance solely from self-state features. These measures characterize general aspects of navigation behavior that are not tied to specific scene layouts.

Further, we conduct a Cross-Environment Reward Consistency experiment. Specifically, we randomly split the available training scenes into two disjoint subsets, using 80% of the scenes to train and estimate self-state-conditioned policy rewards and reserving the remaining 20% as held-out unseen scenes.

For each self-state SS, we compare the relative rankings of different policies between the estimation and held-out sets using Pairwise Ranking Agreement (PRA) and Kendall's rank correlation coefficient (τb\tau_b). PRA measures the proportion of policy pairs whose relative rankings remain consistent across the two sets:

\frac{1}{\lvert\mathcal P_S\lvert}\sum_{(\pi_a,\pi_b)\in\mathcal P_S} \mathbb I \left[ \operatorname{sign}(\Delta_{\mathrm{train}})= \operatorname{sign}(\Delta_{\mathrm{heldout}}) \right],$$ where $$\Delta_{\mathrm{train}}= R_{\mathrm{train}}^{\pi_a}(S)-R_{\mathrm{train}}^{\pi_b}(S),$$ and $\Delta_{\mathrm{heldout}}$ is defined analogously using the held-out data. Here, $\mathcal P_S$ denotes the set of comparable policy pairs under self-state $S$. We additionally employ Kendall's $\tau_b$ to measure the overall rank correlation between the two policy rankings. Unlike PRA, which measures the proportion of consistent pairwise comparisons, Kendall's $\tau_b$ accounts for both concordant and discordant pairs while handling tied rankings. It is defined as: $$\tau_b = \frac{C-D} {\sqrt{ (C+D+T_{\mathrm{train}}) (C+D+T_{\mathrm{heldout}}) }},$$ where $C$ and $D$ denote the numbers of concordant and discordant policy pairs, respectively, while $T_{\mathrm{train}}$ and $T_{\mathrm{heldout}}$ denote the numbers of pairs tied only in the corresponding rankings. Higher values of both metrics indicate stronger consistency between the policy rankings across the two subsets. **Table 1. Cross-Environment Reward Consistency** | Method | Self-state | PRA ↑ | Kendall's $\tau_b$ ↑ | |---|---|---:|---:| | ApexNav | $S_1$ | 0.86 | 0.73 | | ApexNav | $S_2$ | 0.80 | 0.60 | | ApexNav | $S_3$ | 0.74 | 0.57 | The results demonstrate that self-state-conditioned reward estimates exhibit consistent policy rankings on held-out scenes. Although the consistency varies across self-states, both metrics remain consistently positive, indicating that relative policy performance estimated from 80% of the training scenes can generalize to the held-out 20%. ### 2-2. Assignment of Task Completion Rewards to Trajectory Fragments --- We apologize that the original manuscript did not explicitly describe how the episode-level success reward is assigned to individual trajectory fragments. In our implementation, the success reward is shared by all trajectory fragments within a successful episode, providing episode-level task outcome feedback. The individual contributions of each fragment are further characterized by the step penalty and alignment reward. The reward combines a shared global task outcome signal with fragment-specific behavioral feedback grounded in the discovered skill representations. Our objective is to evaluate the overall policy performance under different self-states by combining global task completion feedback with local behavioral signals. To further examine the contribution of the three reward components, we conduct a reward component ablation study. **Table 3. Ablation of Reward Components on HM3D-v1 of VLFM+ApexNav .** | Reward Variant | Success | Step | BasicSkill-guided Alignment | SR (%) ↑ | SPL (%) ↑ | |---|:---:|:---:|:---:|---:|---:| | Success only | ✓ | | | 58.0 | 30.6 | | Success + Step | ✓ | ✓ | | 57.6 | 30.4 | | Success + Alignment | ✓ | | ✓ | 59.8 | 33.1 | | Step + Alignment | | ✓ | ✓ | 57.3 | 30.1 | | **Full Reward (Ours)** | **✓** | **✓** | **✓** | **60.4** | **33.2** | We will revise Section 3.5.1 and Appendix A.4 to explicitly describe all three reward components, clarify how the BasicSkill reference motion representation is constructed from the learned behavioral basis space and activation vector, and explain the episode-level success reward assignment.

(W3) Overly Complicated Methodology with Multiple Engineering Components


1. A Unified Principle Behind SelfSkill. The central objective of SelfSkill is to characterize the behaviors of agents under current embodied conditions and further perform cross-agent improvement . SelfSkill represents behaviors through the activation of interpretable behavioral factors under different self-states. The components of SelfSkill serve four closely connected purposes: (1) Embodied condition characterization: Self-states describe the agent's current movement, perception, and interaction conditions, providing the context in which behavioral patterns are characterized. (2) Behavioral factor representation: The behavioral basis space provides a shared vocabulary for representing environmental factors that may influence navigation behavior. (3) Behavioral activation inference: Trajectory observations are used to infer how different factors contribute to the agent's actual motion under each self-state. (4) Skill utilization: The resulting skill representations are used to evaluate complementary policy capabilities and guide subsequent policy enhancement.

2. Targeted Ablation of Major Design Components.

We compare the full SelfSkill with the following variants:

  • w/o Self-state: We aggregate trajectory fragments without distinguishing different self-states, thereby examining whether embodied condition awareness is necessary for recovering meaningful behavioral patterns.
  • Simplified Basis: We replace the full behavioral basis space with a goal-only basis that retains goal-related semantic factors, examining whether environmental factors beyond direct goal-related information contribute to behavioral characterization.
  • w/o LLM Prior: We remove the LLM-generated activation prior while retaining the motion reconstruction objective and sparsity constraint, examining the contribution of semantic guidance.
  • w/o Regularization: We remove the sparsity regularization while retaining the motion reconstruction objective and prior-consistency term, examining its contribution to obtaining compact behavioral explanations.

We also evaluate these variants using Behavior-Activation Consistency (BAC, its detailed definition is at the end of the discussion) and motion reconstruction error. BAC measures the agreement between the behavioral factors inferred from actual motion and those supported by the learned activation patterns, while reconstruction error measures how accurately the inferred behavioral representations explain the observed motion.

Table 1. Ablation of the Major Components on HM3D-v1 of ApexNav+BasicSkill.

VariantBAC ↑Reconstruction Error ↓SR (%) ↑SPL (%) ↑
w/o Self-state0.420.6657.030.4
Simplified Basis0.080.9438.221.6
w/o LLM Prior0.530.4958.231.6
w/o Regularization0.550.4860.533.0
Full SelfSkill0.590.3762.233.7

Full SelfSkill achieves the best performance across all four metrics. Removing self-state information, simplifying the behavioral basis, or removing the LLM prior and regularization leads to performance degradation. Notably, simplifying the basis causes the largest drop, highlighting the importance of an expressive behavioral representation. These results validate the contributions of the major components of SelfSkill.


The definition od Behavior-Activation Consistency. We characterize the actual behaviors of agents mm under state SS. We evaluate the behaviors of fragments toward or away from the environmental instances associated with different behavioral factors. For object, frontier, and open-space factors, we measure the change in geodesic distance between the fragment endpoints and the corresponding spatial region. For an basis instance jj with spatial region Rk,jR_{k,j}: rk,j=dgeo(pkstart,Rk,j)−dgeo(pkend,Rk,j)ℓk+ϵr_{k,j}=\frac{d_{geo}(p_k^{start},R_{k,j})-d_{geo}( p_k^{end},R_{k,j})}{\ell_k+\epsilon}, where dgeo(p,R)d_{geo}( p,R) denotes the geodesic distance from position $ p$ to region RR, and ℓk\ell_k is the geodesic distance between the fragment endpoints. A positive value indicates progress toward the corresponding instance. For the Boundary factor, which represents spatial avoidance rather than attraction: rk,j=dclear(pkend,Bk,j)−dclear(pkstart,Bk,j)ℓk+ϵr_{k,j}=\frac{d_{clear}( p_k^{end},B_{k,j})-d_{clear}( p_k^{start},B_{k,j})}{\ell_k+\epsilon}, where dclear(p,B)d_{clear}(p,B) measures the spatial clearance between position pp and boundary region BB. We classify the behavior tpyes factf^{act} toward its available basis instances through fkact=arg⁡max⁡jrk,jf_k^{act}=\arg\max_{j}r_{k,j}.

For each fragment, we identify the behavioral basis instances. We then calculate the cosine similarity between the observed motion direction vkv_k and each basis direction dk,jd_{k,j}: Ck,j=vk⊤dk,j∥vk∥2∥dk,j∥2.C_{k,j} = \frac{v_k^{\top} d_{k,j}}{\lVert v_k\lVert_2 \lVert d_{k,j}\lVert_2}. For each basis instance, we calculate its contribution by combining its previously inferred activation coefficient with its directional alignment: $Q_{k,j} =\alpha_{j}^{m,S} \max(0,C_{k,j}).$ The behavioral factor with the highest contribution is regarded as the behavior discovered by SelfSkill: f^kskill=arg⁡max⁡jQk,j.\hat f_k^{skill}=\arg\max_{j}Q_{k,j}.

We then calculate BAC by comparing the discovered factor with the actual behavioral factor: BAC(m,S)=∑kI[f^kskill=fkact]Nall.\mathrm{BAC}(m,S)=\frac{\sum_{k}\mathbb{I}\left[\hat f_k^{skill}=f_k^{act}\right]}{N_{all}}.

Reviewer_dpQgOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SelfSkill, a method to understand and improve the skills of embodied agents. It studies how agents move in different situations, such as when they are stuck or have seen the target. It uses movement data and a language model to find which factors may guide their actions. These results help compare agents and use their strengths to improve performance. Experiments on object navigation and mobile manipulation show that the method can improve existing agents.

Critical strengths
  1. The paper defines skills by how an agent combines and uses environmental factors in a specific state. This gives complex behavior a clear structure, making it easier to compare agents and identify their strengths in different situations.
  2. The skill analysis is also used to improve agents. SelfSkill can choose a stronger policy for the current state or use skills from successful agents to guide another policy. This provides a practical way to use the different strengths of existing agents without retraining their original policies.
  3. The experiments show clear gains for different agents. Adding BasicSkills to VLFM on HM3D-v2 raises SR from 63.6% to 73.7% and SPL from 32.5% to 34.4%. These results support the value of skill guidance across different policies and environments.
Critical weaknesses

The paper gives stronger evidence that the skills are useful than that they correctly explain an agent’s behavior. Table 3 measures motion prediction, while Figure 3b and Appendix C mainly show the skill weights. These results do not directly test whether the weights correctly explain an agent’s behavior. Testing agents with known behavior rules would provide stronger evidence. 2. The fusion results do not clearly support the need for skill decomposition. In Section 3.5.2, policy fusion selects policies by their average reward in each state. This selection appears possible using only a state–reward table, without the skill weights. The gains therefore support state-based policy selection, but leave the added value of the proposed skill representation unclear.

Questions for authors
  1. Can SelfSkill recover known behavior differences between agents, such as following frontiers versus moving toward open space? This would help check whether the skill weights explain the agents’ behavior, rather than only predict their movement.
  2. For skill-guided intervention, how does the full method compare with using only the LLM prior, or fitting one set of weights across all self-states? Using the same training trajectories where needed would help show what the state-conditioned skill inference adds.
3 comments

We thank the reviewer for raising these important concerns.

(W1 & Q1) Behavioral Explanation


(1) Actual Behavioral Analysis. In the section, we characterize the actual behaviors of agents mm under state SS. We evaluate the behaviors of fragments toward or away from the environmental instances associated with different behavioral factors. For object, frontier, and open-space factors, we measure the change in geodesic distance between the fragment endpoints and the corresponding spatial region. For an basis instance jj with spatial region Rk,jR_{k,j}: rk,j=dgeo(pkstart,Rk,j)−dgeo(pkend,Rk,j)ℓk+ϵr_{k,j}=\frac{d_{geo}(p_k^{start},R_{k,j})-d_{geo}( p_k^{end},R_{k,j})}{\ell_k+\epsilon}, where dgeo(p,R)d_{geo}( p,R) denotes the geodesic distance from position $ p$ to region RR, and ℓk\ell_k is the geodesic distance between the fragment endpoints. A positive value indicates progress toward the corresponding instance. For the Boundary factor, which represents spatial avoidance rather than attraction: rk,j=dclear(pkend,Bk,j)−dclear(pkstart,Bk,j)ℓk+ϵr_{k,j}=\frac{d_{clear}( p_k^{end},B_{k,j})-d_{clear}( p_k^{start},B_{k,j})}{\ell_k+\epsilon}, where dclear(p,B)d_{clear}(p,B) measures the spatial clearance between position pp and boundary region BB.

We classify the behavior tpyes factf^{act} toward its available basis instances through fkact=arg⁡max⁡jrk,jf_k^{act}=\arg\max_{j}r_{k,j}. The behavioral proportion of factor type ff is calculated as: Pfact(m,S)=NfactNallP_{f^{act}}(m,S)=\frac{N_{f^{act}}}{N_{all}}

(2) Skill-based Behavioral Analysis. Activation Support of behaviors: For each fragment, we identify the behavioral basis instances. We then calculate the cosine similarity between the observed motion direction vkv_k and each basis direction dk,jd_{k,j}: Ck,j=vk⊤dk,j∥vk∥2∥dk,j∥2.C_{k,j} = \frac{v_k^{\top} d_{k,j}}{\lVert v_k\lVert_2 \lVert d_{k,j}\lVert_2}. We examine whether the behavioral factor identified from the observed displacement receives support from the previously inferred activations: ASfact(m,S)=∑kI[αk,fkactm,S>δ∧Ck,fkact≥τ]Nfact,AS_{f^{act}}(m,S)=\frac{\sum_{k}\mathbb{I}\left[\alpha_{k,f_k^{act}}^{m,S}>\delta\land C_{k,f_k^{act}}\geq\tau\right]}{N_{f^{act}}}, where τ\tau is a predefined directional-alignment threshold.

Behavior-Activation Consistency: To further quantify whether the learned activations explain the observed behavioral differences, we introduce Behavior-Activation Consistency. For each basis instance, we calculate its contribution by combining its previously inferred activation coefficient with its directional alignment: $Q_{k,j} =\alpha_{j}^{m,S} \max(0,C_{k,j}).$ The behavioral factor with the highest contribution is regarded as the behavior discovered by SelfSkill: f^kskill=arg⁡max⁡jQk,j.\hat f_k^{skill}=\arg\max_{j}Q_{k,j}. We then calculate BAC by comparing the discovered factor with the actual behavioral factor: BAC(m,S)=∑kI[f^kskill=fkact]Nall.\mathrm{BAC}(m,S)=\frac{\sum_{k}\mathbb{I}\left[\hat f_k^{skill}=f_k^{act}\right]}{N_{all}}.

(W1 & Q1) Behavioral Explanation


(3) Quantitative results. We conduct the analysis across three representative self-states, all with Couch as the navigation goal in the next behavioral explanation: (1) S1S_1 : Stuck, Navigate to Couch and Target not observed, (2) S2S_2 : Normal, Navigate to Couch and Target not observed, and (3) S3S_3 : Normal, Navigate to Couch and Target observed. We examine the four behavioral factor types ff in SelfSkill: Toward object cluster about seating area f1f^1, Toward frontier f2f^2, Toward open-space f3f^3, and Away from boundary f4f^4.

Table 1. Activation Support ASfactAS_{f^{act}} of behaviors under different agents mm and self-states SS, and Behavior-Activation Consistency for different agents mm. We take the geometric mean of multiple related instances.

MethodSelf-stateαf1m,S\alpha^{m,S}_{f^1}αf2m,S\alpha^{m,S}_{f^2}αf3m,S\alpha^{m,S}_{f^3}αf4m,S\alpha^{m,S}_{f^4}Pf1P_{f^1}Pf2P_{f^2}Pf3P_{f^3}Pf4P_{f^4}ASf1AS_{f^1}ASf2AS_{f^2}ASf3AS_{f^3}ASf4AS_{f^4}BACBAC
VLFMS1S_10.020.040.100.180.010.030.320.460.000.390.600.570.49
VLFMS2S_20.030.160.150.200.040.320.200.170.750.560.650.820.51
VLFMS3S_30.200.100.110.320.120.240.170.240.890.920.780.530.53
ApexNavS1S_10.000.000.210.100.000.140.340.44–0.000.800.500.50
ApexNavS2S_20.050.250.120.110.020.420.200.190.900.960.930.790.60
ApexNavS3S_30.260.120.060.240.250.090.170.220.960.950.480.890.55

Table 1 reveals distinct behavioral preferences between VLFM and ApexNav across different self-states, consistent with their underlying navigation designs. Under the stuck state S1S_1, both agents primarily exhibit open-space-seeking and boundary-avoiding behaviors, reflecting their reliance on local geometric cues to recover effective navigation. Under normal motion without observing the target S2S_2, both show stronger frontier-following tendencies, with ApexNav exhibiting a more pronounced preference (0.42 vs. 0.32). This is consistent with its adaptive exploration strategy, which switches between semantic and geometry-based exploration according to the availability of semantic cues.

A more notable difference emerges when the target is observed $ S_3$. ApexNav exhibits a stronger object-cluster-directed preference (0.25 vs. 0.12) and a weaker frontier-following tendency (0.09 vs. 0.24) than VLFM. This is consistent with ApexNav's target-centric semantic fusion, which maintains reliable target information across observations and enables more direct target-oriented navigation. In contrast, VLFM retains a more prominent frontier-related behavioral tendency, reflecting its frontier-based exploration design. These differences are also reflected in the inferred activation patterns, particularly the higher object-cluster activation of ApexNav under S3S_3 (0.26 vs. 0.20).

Overall, these results demonstrate that SelfSkill captures both state-dependent behavioral variations within individual agents and distinct functional preferences across different agents. ApexNav also achieves slightly higher BAC scores across all three self-states, suggesting a stronger correspondence between its inferred skill activations and observed behaviors in these settings.

(W2) Clarification of the Reward Formulation


SelfSkill discovers interpretable behavioral strategies by constructing a behavioral basis space and inferring the activation patterns of different behavioral factors under various self-states. These learned representations also provide the foundation for evaluating candidate policies in the subsequent skill-aware policy fusion.

We apologize that the original manuscript did not fully describe the reward formulation used in our experiments, which consists of three complementary components: task completion reward, step penalty, and BasicSkill-guided motion alignment:

R_k^{\mathrm{success}} + R_k^{\mathrm{step}} + \lambda_a R_k^{\mathrm{align}}.$$ The first two components evaluate task outcomes and execution cost, respectively. More importantly, the alignment component directly connects policy evaluation with the behavioral representations learned by SelfSkill. Specifically, BasicSkill denotes a set of skills derived from optimal paths collected on the training dataset. To construct a reference motion representation, we leverage the behavioral basis instances and the BasicSkill activation vector $\boldsymbol{\alpha}_k^{\mathrm{Basic}}$ learned by our method: $$\hat{\mathbf v}_k^{\mathrm{Basic}}=\mathbf F_k \boldsymbol{\alpha}_k^{\mathrm{Basic}},$$ where $\mathbf F_k$ represents the behavioral basis instances associated with fragment $k$. For a fragment generated by policy $\pi$, we extract its actual motion representation $\mathbf v_k^\pi$. The alignment reward is then defined as: $$R_k^{\mathrm{align}}= \frac{ (\mathbf v_k^\pi)^\top \hat{\mathbf v}_k^{\mathrm{Basic}} }{ \lVert\mathbf v_k^\pi\lVert_2 \lVert\hat{\mathbf v}_k^{\mathrm{Basic}}\lVert_2+\epsilon }.$$ This component measures the consistency between the actual behavioral tendency of a candidate policy and the reference behavioral patterns represented by BasicSkill in the SelfSkill space. Importantly, this establishes a direct connection between skill discovery and policy evaluation: the behavioral basis space and activation representations learned during skill discovery are reused to construct a skill-based reference for evaluating candidate policies. Therefore, the alignment reward is not an independent, manually specified motion heuristic, but an evaluation signal grounded in the behavioral representations learned by SelfSkill. We will explicitly clarify the complete reward formulation and its connection to skill discovery in the revised manuscript. ### (Q2) Comparison on method variants --- We compare three variants using the same training trajectories, initial behavioral basis space, and downstream intervention procedure: **(1) LLM Prior Only.** We directly use the activation means predicted by the LLM. **(2) Shared Weights Across Self-states.** We fit a single shared activation pattern using training trajectory fragments from all self-states, rather than inferring separate activation patterns for individual self-states. **(3) Full SelfSkill.** We retain the complete formulation, jointly leveraging trajectory reconstruction and the LLM prior to infer a separate activation pattern for each self-state. For a fair comparison, all three variants will use the same training trajectories and behavioral basis space. **Table 2. Comparison about LLM Prior Only, Shared Weights and Full SelfSkill on HM3D-v1. Fusion and Intervention represent the downstream methods: VLFM+ApexNav and ApexNav+BasicSkills, respectively.** | Variant | (Fusion) SR (%) ↑ | (Fusion) SPL (%) ↑ | (Intervention) SR (%) ↑ | (Intervention) SPL (%) ↑ | |---|---|---:|---:|---:| |LLM Prior Only|58.9|30.8|60.2|32.7| | Shared Weights | 55.2 | 26.9 | 57.0 | 30.4 | | Full SelfSkill | 60.4 | 33.2 | 62.2 | 33.7 | Table 2 demonstrates that Full SelfSkill consistently achieves the best performance in both downstream settings, including policy fusion and skill-guided intervention. Compared with LLM Prior Only, it improves SR/SPL by 1.5/2.4 percentage points for fusion and 2.0/1.0 points for intervention, showing the benefit of incorporating trajectory-based behavioral evidence beyond semantic priors. Compared with Shared Weights, the improvements are more pronounced, with SR increasing by 5.2 points in both settings. This indicates that a single shared activation pattern is insufficient to capture the diverse behavioral requirements across self-states. Overall, these results validate the complementary contributions of trajectory-based inference and self-state-specific skill representations to effective policy enhancement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.