acceptodds
Under review as a conference paper at ICLR 2027

LEARNING WHICH CHANGES MATTER: PREFERENCE-CONDITIONED SKILL DISCOVERY

Abstract

Unsupervised Skill Discovery (USD) aims to autonomously learn a diverse set of reusable skills without relying on extrinsic rewards. However, existing methods mainly encourage skills to be diverse or distinguishable, which can result in behaviors that are easy to differentiate but lack behavioral significance. We hypothesize that leveraging the extensive knowledge encoded in pretrained large language models (LLMs) can help identify behaviorally consequential skills based on their executions. In this sense, we introduce Preference-Conditioned Skill Discovery(PCSD), a skill discovery framework that incorporates task-agnostic preference guidance to discover skills with greater behavioral significance. PCSD decomposes skill executions into factor-specific behavioral changes and learns preferences separately for each factor to avoid bias arising from their different physical roles. PCSD defines a preference-conditioned mutual-information objective to prioritize behaviorally consequential skill executions, and provides a variational interpretation of this objective under a induced distribution. Across multiple environments, we show that PCSD assigns higher preference to coherent and behaviorally consequential skill executions. PCSD consequently learns skill repertoires that are more useful for downstream tasks and achieves improved performance over existing skill discovery baselines.

Primary area:
reinforcement learning
Keywords:
Skill Discovery, Preference Learning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_S3N2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Uses factored skills where the skill affects particular factors, and then applies preference scores to prioritize more consequential executions. The preferences between different skills are determined by the preference measure. Preferences are computed using the Bradley-Terry model and modulate the mutual information lower bound by multiplying it by the sum of skill rewards for a trajectory.

Critical strengths

The paper is generally straightforward to follow and applies a straightforward application of preferences to unsupervised skill discovery.

Critical weaknesses

Language agents have the capability of creating scripted skills directly, which raises the question: if we are using the language model for the preference, why are we using an unsupervised skill discovery model for the skills and not also using the language model for the skills? The best reasoning would be that the model is unable to do skill learning but is good as a discriminator, which may be true, but is not evaluated in this work.

The figures (especially Figure 3) do a poor job of demonstrating what is being added in the context of the experiment by the preferences given by the language model. In particular, it looks like in the longer sequential multiparticle is where the best learning happens, but it is not as clear what kinds of preferences would be necessary in those tasks, nor is there any visualization or figure to represent this in the main paper.

It is not clear how to interpret the downstream reward, since the numbers appear to be quite small. The main paper also appears to be missing the performance graph for iGibson wipe, which is the only "real" domain, at least insofar as it is a realistic simulator.

Questions for authors

See Weaknesses

3 comments

We thank the reviewer for finding our work relevant and impressive and for giving valuable feedback. We updated the paper with expanded related-work discussion (Section 2), paired Multi-Particle skill visualizations (Appendix F.1, Figure 7), explicit downstream reward-scale definitions (Section 5.2, Appendix E), and 7-method results including iGibson Wipe (Figure 3(f), Table 1).

1. Why Use USD + LLM Preferences Instead of Scripting Skills Directly?

Your intuition captures the core reason accurately, which we make explicit in Section 2 (p. 3):

  • Continuous low-level dynamics vs. pre-engineered APIs: Language agents that output scripted skills (SayCan, Code as Policies) do not synthesize raw continuous motor control; instead, they compose pre-engineered perception and motion-primitive APIs (e.g., inverse kinematics, waypoint or grasp controllers). Our environments expose raw continuous joint torques, coupled multi-entity forces, and bimanual contact control without hand-crafted APIs, where LLMs cannot directly script stable closed-loop actions due to unmodeled contact dynamics (Language to Rewards).
  • Task-agnostic repertoire discovery: Even LLM reward-code synthesis (Eureka, Language to Rewards) requires task descriptions, simulator source code, or task-fitness feedback. By contrast, PCSD operates in strictly task-agnostic pretraining without task goals or simulator internals: the LLM acts where foundation models excel—as a comparative judge of behavioral coherence over observed trajectories (RL-VLM-F, Motif)—while RL + mutual information acquires closed-loop policies spanning a diverse skill repertoire reusable across unseen tasks.

2. What Preferences Matter in Multi-Particle and Why Gains Grow at L=8,9L=8, 9

  • Task-agnostic preference criteria (Appendix G.3): During pretraining, the LLM receives no information about sequential orders, food, or poison. Given 5 sparse keyframes of an agent's motion relative to its paired landmark, the prompt favors coherent, purposeful behaviors—smooth approach, decisive landmark entry, sustained holding, or clean traversal—over aimless jitter, loops, or drifting into empty arena boundaries.
  • Visualizing the skill difference (Appendix F.1, Figure 7): We added a paired comparison between DUSDi and PCSD across all 5 discrete factor skills from identical resets, plotting 2D paths and landmark distance di(t)d_i(t). Without preferences, DUSDi treats every separable motion equally and drives multiple skills outward to slide along arena boundaries, leaving sparse coverage near the landmark. PCSD suppresses boundary wandering and allocates multiple skills to landmark entry and sustained holding (di<0.6d_i < 0.6), alongside controlled standby and retreat (Figures 7–8).
  • Why gains grow on Sequential L=8,9L=8, 9 (Table 1, Figure 3(a–c)): Sequential tasks require LL agents to successively reach and hold inside their landmarks while others avoid premature entry. Because DUSDi learns fewer reliable hold/standby skills per factor, chaining LL accurate skills degrades rapidly as LL grows: even under zero-shot random skill sampling (Table 1), PCSD more than doubles DUSDi's positive-return access at L=8L=8 (6.20%6.20\% vs. 3.10%3.10\%) and L=9L=9 (2.50%2.50\% vs. 1.00%1.00\%), directly driving the larger downstream gains in Figure 3(b, c).

3. Downstream Reward Scales and iGibson Wipe Performance

  • Interpreting return magnitudes (Section 5.2, Appendix E, Eq. 17): Following standard hierarchical evaluation (H=50H=50 steps per skill), downstream wrappers average each segment's step rewards by 1/H1/H, rhigh,t=1H∑h=1Hrenv,t+hr_{\mathrm{high},t} = \frac{1}{H}\sum_{h=1}^{H} r_{\mathrm{env},t+h} (Eq. 17), keeping PPO value scales consistent. Episodic returns in Figure 3 sum these 1/H1/H-averaged segment rewards over T/HT/H skill selections (raw cumulative environment rewards are 50×50\times larger). For instance, in MoMa2D (2020 selections, raw step reward ≤+2\le +2), the return ceiling is 40.040.0 (Figure 3(g)), which PCSD nearly saturates; in Multi-Particle Sequential, completing all stages grants +10+10 per remaining segment, yielding returns above 100100 (Figure 3(a–c)). Full definitions for all 12 tasks are in Appendix E.
  • iGibson Wipe performance (Figure 3(f), Table 1): Figure 3(f) in the main text reports full learning curves on iGibson Wipe across all 7 methods (including CDP and HaSD), where PCSD achieves the highest return by a clear margin, corroborated by a 16.00%16.00\% zero-shot mark-removal rate in Table 1 (vs. 2.50%2.50\% for DUSDi).
Reviewer_S3N2OpenReview ↗

Response to Authors

I thank the authors for their lengthy response, and I believe that the additions will improve the paper, though none of them significantly change my view of the core weaknesses. In particular, the logic, while sound, is ungrounded in why not just use an LM to script behaviors, and would be stronger with a direct comparison. The Figure 3 weaknesses appear to still be present, and the description of the Gibson wipe is still vague. I will maintain my score for the original reason.

Thank you very much for reading our response, for noting that the additions improve the paper, and for maintaining your positive assessment!

Following your helpful feedback, we have also updated the manuscript with two targeted clarifications:

  1. Visualizing preferred Multi-Particle behaviors (Section 5.3 & Appendix F.1, Figure 7): We added Figure 7 to directly show concrete 5-keyframe factor-wise preference pairs during task-agnostic pretraining (preferring coherent landmark settling, controlled interior retreat, and persistent directional transit over transient pass-by drift, boundary wall-pinning, and erratic reversals) alongside how a downstream controller composes them.
  2. Clarifying iGibson Wipe (Appendix E): We expanded the iGibson Wipe description to detail the episode structure and the three physical components of its downstream reward (base workspace positioning, wiping-hand proximity/posture guidance, and dirt-mark removal bonus).

We also appreciate your suggestion regarding an empirical comparison with LLM-scripted control as a valuable direction for future work. Thank you again for your time and constructive feedback!

Reviewer_RH5VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a preference-conditioned skill discovery method. Based on the factorization of environmental states and their corresponding skill components, the method uses an LLM to compare multiple skill executions within the same factor to obtain preferences, and then uses the preference scores to weight the corresponding transition-level skill mutual information rewards.

Critical strengths

Major Strengths

Introducing LLMs into skill discovery is very valuable for unsupervised RL.

Minor Strengths

The method is clearly presented.

Critical weaknesses
  1. The paper uses an LLM, while all the skill discovery methods used for experimental comparison are unsupervised skill discovery methods without LLMs. This makes the experimental comparisons unfair. The proposed method should be compared with recent SOTA methods such as LGSD and HaSD to better demonstrate its effectiveness.

  2. The state-factor decomposition adopted in the paper, as well as the combination of preference learning and skill discovery, have already been explored in previous work, such as DUSDi and HaSD. This limits the novelty of the paper, and the independent contribution of the proposed method remains unclear.

Questions for authors

1 Can the authors provide additional experimental comparisons with recent methods such as LGSD [1] and HaSD [2]?

[1] Rho, Seungeun, et al. "Language Guided Skill Discovery." International Conference on Learning Representations, 2025.

[2] "Human-Aligned Skill Discovery: Balancing Behaviour Exploration and Alignment." AAMAS, 2025, pp. 1025–1033.

2 On some tasks, the proposed method achieves performance very close to DUSDi, which does not use an LLM. Can the authors provide further analysis of these results?

3 If the proposed method outperforms existing LLM-based skill discovery methods, does the improvement mainly come from preference learning or the state-factor decomposition inherited from DUSDi? How would other methods, such as LGSD, perform when combined with this factorization, compared with the proposed method?

3 comments

We thank the reviewer for recognizing the value of introducing LLM priors into USD and the clarity of our presentation and for giving valuable feedback. In the revised paper, we added LLM/preference-guided baselines across all 5 environments and 12 downstream tasks (Sections 5.1–5.3, Figures 3–4, Table 1).

1. Comparisons with LLM/Preference-Guided Baselines (W1 & Q1)

We added official implementations of HaSD [AAMAS 2025] and CDP across all 5 environments (HalfCheetah, Ant, Multi-Particle, MoMa2D, iGibson) and all 12 tasks (Figure 3, Table 1, Appendices B–C), under the same LLM (deepseek-flash) and 2,000-pair budget as PCSD. PCSD consistently outperforms both on downstream control (Figure 3) while discovering broader, more purposeful skill repertoires (Figure 4, Table 1).

Why HaSD and CDP rather than LGSD: (1) Representation: LGSD captions single raw states, which does not naturally extend to coupled multi-entity coordinates (Multi-Particle) or contact-force states (iGibson Wipe) without hand-crafted state-to-text rules, whereas PCSD, HaSD, and CDP all compare trajectory segments; (2) Budget parity: LGSD queries the LLM online at every RL step, whereas HaSD and CDP allow a fair comparison under a bounded 2,000-pair budget; (3) Reproducibility: LGSD has no public official code, whereas HaSD and CDP provide official implementations.

2. Core Contribution Beyond DUSDi and HaSD (W2)

Our core goal is to discover a repertoire of behaviorally meaningful skills—learning which state changes matter rather than merely maximizing statistical distinguishability. Simply combining factorization with existing preference methods fails to achieve this, motivating two designs:

  1. Factorized Preference Learning (Section 4.2, Figure 1): Whole-rollout preferences (HaSD) assign one score to an entire rollout, conflating meaningful progress in one factor with irrelevant motion in others. PCSD evaluates context-anchored factor summaries Ξi=(st,κ(τi))\Xi^i=(s_t,\kappa(\tau^i)) (Eqs. 5–6) to isolate meaningful changes per factor, specializing cleanly to unfactorized settings when N=1N=1.
  2. Prior-Preserving Preference-Induced MI (Section 4.3, Lemma 1): HaSD adds a preference bonus onto an MI reward, overriding discriminator diversity and collapsing skills onto one favored direction (Figure 4). PCSD instead defines a preference-induced MI objective JPCSD=∑i=1NIPi⋆(Zi;Yi)\mathcal{J}_{\mathrm{PCSD}}=\sum_{i=1}^N I_{P_i^\star}(Z^i;Y^i) (Eqs. 3–4, 7–9) via a null-outcome gate. Lemma 1 proves this preserves the uniform skill prior Pi⋆(Zi)=p(Zi)P_i^\star(Z^i)=p(Z^i), so multiplicative gating concentrates capacity on meaningful behaviors without sacrificing diversity.

3. Why PCSD and DUSDi Overlap on Certain Tasks (Q2)

On simpler tasks—Multi-Particle Sequential (L=7L=7) and Food & Poison (Figure 3(a, d, e))—both methods already approach the return ceiling once factorization decouples the agents (Appendix E), as coarse directional skills suffice. Under longer horizons, tighter precision, or contact constraints (Sequential L=8,9L=8,9, MoMa2D, iGibson Wipe, HalfCheetah, Ant; Figure 3(b, c, f–l)), DUSDi wastes skill capacity on easily distinguishable yet unhelpful behaviors like boundary wandering (Appendix F.1, Figure 7) or local posture shifts (Figure 4), whereas PCSD concentrates skills on sustained landmark holding, surface sweeping, and locomotion (Table 1).

4. Preference vs. Factorization & Factorizing Other LLM Methods (Q3)

  • Disentangling both components: Fixing factorization (N>1N>1): Sharing DUSDi's architecture (n=0n=0), scaling preference pairs from 0→2,0000\to 2{,}000 steadily improves return (Figure 6) and beats DUSDi on complex tasks (Figure 3(b, c, f–h)). Removing factorization (N=1N=1): On unfactorized HalfCheetah and Ant (Figure 3(i)–(l), Figure 4), PCSD still outperforms all baselines, confirming preference-conditioned MI independently drives gains.
  • Factorizing other LLM methods: Our N=1N=1 tasks (Figure 3(i)–(l), Figure 4) directly test within-factor discovery on an identical single-factor space: HaSD collapses onto one direction from its additive bonus and CDP stays localized, so applying them per factor would still suffer within-factor collapse or localization, whereas PCSD's multiplicative gating (Eq. 4) preserves within-factor diversity. Similarly, factorizing LGSD would still optimize static single-state caption distance rather than trajectory coherence while multiplying per-step LLM calls by NN.

Thank you to the authors for their detailed response. The additional experiments have improved the quality of the paper. However, I still believe that the proposed method is largely a combination of existing approaches. I would have preferred to see some new perspectives on how LLMs can contribute to skill discovery, such as methods for providing intrinsic rewards.

Therefore, I have decided to maintain my rating. Nevertheless, I would also support the paper being accepted because, as the authors mentioned, LGSD has not released its code, and this field indeed lacks works with publicly available code. I also hope that, if the paper is accepted, the authors will provide a well-organized code repository.

Thank you very much for taking the time to review our rebuttal and revised manuscript, and for your support for accepting the paper!

Regarding code availability, we have already provided our code in the supplementary material, and we will promptly update and maintain a well-organized open-source repository.

We also appreciate your perspective on exploring new ways for LLMs to provide intrinsic rewards. Beyond reshaping transition-level intrinsic rewards via multiplicative preference conditioning as in PCSD (Eq. 4), we agree that investigating broader paradigms for LLM-driven intrinsic reward design is a promising direction for future work. Thank you again for your valuable feedback!

Reviewer_93XDOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Preference-Conditioned Skill Discovery (PCSD), a method for learning diverse, meaningful skills without task-specific rewards. It uses LLM-based comparisons to learn preference scores for individual factors of the environment, then uses these scores to weight skill-discovery rewards, encouraging behaviors with more significant consequences. Across eight downstream tasks in three simulated environments, PCSD outperforms existing skill-discovery methods while maintaining broad behavioral diversity.

Critical strengths
  1. The method performs well across several environments, with clear gains over existing baselines, especially on the harder sequential tasks.
  2. The theoretical justification is helpful and gives a principled explanation of the preference-conditioned objective through mutual information.
Critical weaknesses

1- The main idea appears to build on incremental combinations of existing methods for factorized skill discovery and preference-based learning. 2- The paper discusses closely related preference-guided methods, such as HaSD and CDP, in the related work section but does not compare its approach with them experimentally. 3- The writing and the paper’s main figure are not easy to follow and could be revised for clarity.

Questions for authors

1- Could the authors evaluate PCSD on Ant and HalfCheetah, as done in SUSD, to see how well it performs in unfactorized settings? 2- Could the authors compare PCSD with related preference-based skill discovery methods, such as HaSD and CDP? Also see weaknesses.

3 comments

We thank the reviewer for finding our work relevant and impressive and for giving valuable feedback. In the revised manuscript, we added all requested evaluations—unfactorized Ant and HalfCheetah as well as head-to-head comparisons against CDP and HaSD across all five environments—and improved the clarity of our figures and writing.

1. Comparisons with CDP and HaSD (W2 & Q2)

We evaluated CDP (Hussonnois et al., 2023) and HaSD (Hussonnois et al., 2025) using their official codebases under the same pairwise preference budget as PCSD across all five environments and twelve downstream tasks (Sections 5.1–5.3, Figures 3–4, Table 1, Appendices B–C):

  • Downstream control (Figure 3): PCSD consistently outperforms both CDP and HaSD across all twelve tasks spanning locomotion, multi-agent coordination, mobile manipulation, and embodied wiping.
  • Skill diversity and zero-shot access (Figure 4, Table 1): In locomotion (Figure 4), CDP stays localized or unidirectional while HaSD collapses skills toward one favored direction; PCSD preserves directional diversity alongside sustained motion. In multi-factor domains (Table 1), whole-trajectory preferences cannot decouple individual factors, giving near-zero access on Multi-Particle Sequential tasks and stagnating downstream on MoMa2D and iGibson Wipe (Figure 3).

2. Unfactorized Settings: Ant and HalfCheetah (Q1)

Following SUSD, we evaluated PCSD and all six baselines in unfactorized Ant and HalfCheetah (N=1N=1) on HalfCheetah Goal (easy/hard) and Ant Goal (easy/hard) (Sections 5.2–5.3, Figure 3(i)–(l), Figure 4). PCSD achieves the highest returns across all four tasks. As shown in Figure 4, preference-free baselines (DIAYN, DUSDi, SUSD) settle for minor posture shifts or early-saturating motion, while CDP and HaSD lose directional diversity; PCSD learns smooth omnidirectional locomotion in Ant and balanced bidirectional running in HalfCheetah.

3. Core Contribution: Discovering a Behaviorally Meaningful Skill Repertoire (W1)

Rather than incrementally combining existing modules, PCSD addresses a fundamental limitation of unsupervised skill discovery: easily distinguishable state changes are not necessarily behaviorally meaningful. Standard USD methods (including DUSDi and SUSD) treat all separable changes equally, wasting skill capacity on local jitter or aimless boundary wandering. Conversely, global preference methods (CDP and HaSD) obscure factor-specific effects in compositional tasks (Figure 1) and pull distinct skills into one favored behavior when preferences act as an additive bonus (Figure 4).

PCSD instead discovers a behaviorally meaningful skill repertoire by redefining which state changes carry skill information: (i) factorized preference learning (Section 4.2) evaluates each factor's behavioral significance independently within its full physical context; and (ii) preference-induced MI maximization (Section 4.3, Lemma 1) multiplicatively conditions local skill information on execution preferences while preserving the skill prior Pi⋆(Zi)=p(Zi)P_i^\star(Z^i)=p(Z^i), rewarding skills only when distinguishable through meaningful changes. Notably, even in unfactorized (N=1N=1) Ant and HalfCheetah (Figure 3(i)–(l), Figure 4)—with zero state factorization—PCSD substantially outperforms DIAYN, METRA, CDP, and HaSD, confirming the standalone value of our core formulation.

4. Writing and Figure Clarity (W3)

We refined Figure 2 and unified notation across Sections 3–4 (removing duplicate equations and specifying the sparse-keyframe summary in Section 4.2). We also added Experimental Setup (Section 5.1), expanded Figure 3 to twelve tasks across five environments, added trajectory diagnostics in Figure 4 and Appendix F.1 (Figure 7), and streamlined the experimental narrative.

We hope these revisions address all your concerns and welcome any further questions.

Thank you for your detailed responses. After considering your rebuttal and the comments from the other reviewers, I've decided to keep my original score, as my concern about Weakness 1 remains unresolved.

Thank you very much for taking the time to read our rebuttal and revised manuscript. We appreciate that our new evaluations and revisions have addressed your empirical and presentation questions. Thank you again for your time and constructive feedback throughout the review process!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

Dear AC,

Thank you for your time in coordinating the evaluation of our submission. Below we provide a concise summary of our core contribution, added experiments, clarifications, and post-rebuttal discussion updates.

Core Conceptual Contribution

Much like LGSD introduces maximizing semantic distance as its guiding principle, PCSD introduces behavioral consequentiality as an inductive bias to discover a behaviorally meaningful skill repertoire—learning which state changes truly matter rather than combining off-the-shelf modules:

  • Prior-preserving MI conditioning: Rather than adding preference bonuses that override the skill prior and collapse diversity (as in HaSD), PCSD combines factorized preference learning with multiplicative MI conditioning (Sections 4.2–4.3, Lemma 1), preserving diversity and outperforming all baselines even without factorization (N=1N=1).
  • Task-agnostic trajectory prompts: Unlike static single-state captions (LGSD) or single-behavior alignment (HaSD), our prompts (Appendix G) evaluate coherent, persistent temporal changes across sparse keyframes without directional or task bias.

Summary of Rebuttal and Discussion

Reviewer 93XD

  • Experiments:
    • Comparisons with CDP and HaSD: Head-to-head evaluations using official CDP and HaSD implementations under the same LLM and 2,000-pair budget across all 5 environments and 12 downstream tasks (Sections 5.1–5.3, Figures 3–4, Table 1, Appendices B–C).
    • Unfactorized Ant and HalfCheetah: Evaluations against all 6 baselines in unfactorized (N=1N=1) HalfCheetah Goal (easy/hard) and Ant Goal (easy/hard) (Figures 3(i)–(l), Figure 4), where PCSD achieves the highest returns across all four tasks.
  • Clarifications:
    • Conceptual contribution: Clarified how PCSD introduces behavioral consequentiality via prior-preserving MI conditioning rather than combining existing modules.
    • Writing and figure clarity: Refined Figure 2, unified notation across Sections 3–4, specified the 5-keyframe summary in Section 4.2, and added Experimental Setup in Section 5.1.

Reviewer RH5V

  • Experiments:
    • Preference-guided baselines: Full comparisons against official HaSD and CDP implementations across all 5 environments and 12 tasks.
    • Disentangling preferences and factorization: Paired the preference-budget ablation at fixed factorization (N=10N=10, Figure 6) with unfactorized locomotion (N=1N=1, Figures 3(i)–(l), Figure 4) to isolate each component's contribution.
  • Clarifications & Discussion Update:
    • Baseline selection & analysis: Explained why trajectory-segment comparisons suit multi-entity and contact states under a bounded 2,000-pair budget with official codebases (unlike per-step state captioning in LGSD), and analyzed why complex tasks expose DUSDi's boundary wandering and HaSD's additive-bonus collapse.
    • Post-rebuttal feedback: Reviewer RH5V noted that the added experiments improved the paper and stated that they would also support acceptance given open code availability; we confirmed our code is in the supplementary material and that the public repository will be promptly updated.

Reviewer S3N2

  • Experiments & Visualizations:
    • Multi-Particle preference and skill visualizations: Added Figure 7 (Appendix F.1, referenced in Section 5.3) to directly illustrate which factor-level behaviors are preferred in Multi-Particle and how they compose in Sequential (L=9), alongside paired DUSDi vs. PCSD skill repertoires in Figure 8 and zero-shot access rates in Table 1.
    • iGibson Wipe downstream performance: Reported full 7-method learning curves on iGibson Wipe in Figure 3(f) and zero-shot mark-removal rates in Table 1.
  • Clarifications & Discussion Update:
    • USD vs. direct scripting & task/reward details: Expanded Section 2 on why raw continuous torque/contact control without primitive APIs favors comparative trajectory preferences paired with MI skill discovery; documented the 1/H1/H (H=50H=50) reward normalization (Section 5.2, Appendix E, Eq. 17) and expanded Appendix E with the physical reward components of iGibson Wipe.
    • Post-rebuttal feedback: Reviewer S3N2 noted the additions improve the paper and maintained their positive rating; the updated Appendix F.1 (Figure 7) and Appendix E directly address their remaining visualization and iGibson Wipe description comments.

Closing Remarks

We hope these experiments, visualizations, and clarifications convey the contribution and empirical strength of our work. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.