- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(−1)
- Soundness
- 2
Read review
Paper summary
This paper studies the relationship between empowerment, state centrality, and downstream adaptation. It provides geometric and hitting-time interpretations of empowerment and establishes an adaptation lower bound in tabular settings under an isotropic Gaussian reward prior. Counterexamples show that higher empowerment does not necessarily imply better adaptation, highlighting limitations in continuous state spaces and under anisotropic rewards.
Critical strengths
- The paper addresses the fundamental question of whether empowerment benefits downstream tasks and has a clear research motivation.
- The paper is generally well written. The geometric interpretations and visualizations in small-scale environments help explain the main results.
Critical weaknesses
-
The centrality result offers limited theoretical novelty. The authors describe Theorem 4.1 as a straightforward extension of Eysenbach et al. (2021), but do not sufficiently explain what new insights its centrality interpretation provides beyond existing information-geometric results.
-
The adaptation lower bound offers limited practical guidance. The authors explicitly acknowledge that the bound does not guarantee consistent rankings between empowerment and adaptation returns. Thus, although the result establishes a conditional theoretical connection, it provides little direct guidance on selecting states or skills to improve downstream adaptation.
-
The empirical evaluation is insufficient. Experiments are limited to small-scale tabular demonstrations, without systematic evaluation of practical skill learning and downstream adaptation. The mismatches in continuous state spaces and under anisotropic rewards also lack empirical validation with learning algorithms, leaving their impact on practical methods unclear.
-
The claims about scalable methods are too strong. The authors argue that scalable empowerment methods must address the mismatch between information geometry and reward geometry. However, practical methods are not inherently required to align these geometries. Without showing that the mismatch limits downstream performance, the paper does not establish why resolving it is necessary.
I would be willing to raise my score if the authors adequately clarify the main concerns above in their response.
Questions for authors
- Beyond applying the existing information-geometric result to the reachable occupancy distributions conditioned on each initial state, what new theoretical insight does Theorem 4.1 provide?
- Beyond the ranking counterexample in Figure 5, are there explicit additional conditions under which comparisons of empowerment can guide the selection of initial states or skill sets? If not, what do the authors consider the main practical implication of Theorem 4.3?
- Could the authors provide empirical evidence beyond small-scale tabular demonstrations by evaluating concrete empowerment-based or mutual-information-based skill-learning methods, examining whether the mismatch occurs in continuous state spaces or under anisotropic rewards and how it affects downstream adaptation?
Other
- Appendix I contains only a heading and no text.
- The readability of Figure 3(b) could be improved.
3 comments
Dear Reviewer wH4z,
Thank you for your thoughtful feedback. We have worked hard to incorporate your feedback with added experiments, revisions, and a stronger, generalized version of the 4.3 bound that applies to a general class of GP priors.
Have our revisions and response sufficiently resolved your questions and concerns? We appreciate your engagement as the end of the discussion phase nears. Thank you.
Thank you for the revision, which appropriately softens the claims on scalable methods and adds helpful diagnostics. However, my main concern regarding empirical depth remains, as the experiments are still limited to exact solvers and controlled toy settings without evaluating practical neural MISL algorithms. Given the improved clarity and rigor, I am willing to marginally raise my score from 2 to 3.
Official Author Response to Reviewer wH4z
Thank you for the thoughtful feedback. We strengthen the original adaptation bound (Theorem I.1) and show a generalized version of the bound for a general class of GP priors using a noise-regularized version of empowerment (Theorem I.9). We have added experiments on adaptation under matched and anisotropic reward priors. Do you have any further concerns after these revisions?
We have revised the text to specifically discuss those related works [1, 2, 3, 4]. There are substantial differences between our theoretical framing and that of [1]. Eysenbach et al. [1] optimize the skill prior and analyze minimax (worst-case) reward initialization against the induced state marginal. We instead index the reachable occupancy polytope by the start state and prove an average-case zero-shot adaptation bound under a Gaussian reward prior. As such, our formulation shows skill/latent-based empowerment can be useful for selecting empowered states, not just partitioning a marginal behavior space effectively.
Theorem 4.3 lower-bounds adaptation from a given state based on that state's empowerment. This does not necessarily imply a ranking over states. Fig. 5 shows why a general ranking claim would be too strong.
With correct priors and γ=0.95, the correlation between effective empowerment and return is 0.97 for center-doorway grids and 0.92 for corner-doorway grids.
Our core contribution in this paper was the geometric and theoretical exploration of a concrete and practical empowerment objective based on skill learning. We have revised the text to clearly state the empirical limitations in §5. A key direction for future work is building and testing scalable methods that maximize this potential empowerment quantity.
We also test anisotropic rewards with equal-trace covariance (rotations) and fixed skills.
Revised manuscript (§5, p. 10): "More broadly, Theorems 4.4 and 4.5 refute universal MI-only guarantees rather than every practical MISL method: smooth encoders, Wasserstein constraints (Metric-aware Abstraction (METRA); Park et al., 2023), and contrastive successor features (Zheng et al., 2025) implicitly regularize against pathological state splitting."
References