acceptodds
Under review as a conference paper at ICLR 2027

Certified Zero-Shot Transfer of Information-Bottleneck Policies

Abstract

Compression is known to improve policy transfer, yet when and why a compressed policy transfers to a new task remains poorly understood. Current empirical approaches evaluate compressed policies on held-out environments, which reveals the outcome of a transfer but not its cause, while theoretical approaches either bound value changes from the two tasks without reference to the policy or relate compression to lost value within a single task. This work aims to understand when and why a policy compressed with the information bottleneck (IB) on a source task transfers to a target task. Given models of both tasks, we answer when with a sufficient condition for near-optimal target return, and why with its terms, each of which names one cause: compression on the source, representation shift, or disagreement between the tasks’ optimal policies. We evaluate the condition in two settings. On tabular grid-worlds, where the IB is solved exactly, it certifies 88.6% of the successful transfers across 13,200 configurations with no false certifications. On neural IB policies trained on MiniGrid images, where the learned representation can itself shift between tasks, it remains sound, and its terms separate failures caused by representation shift from those caused by task disagreement.

Primary area:
reinforcement learning
Keywords:
Information Bottleneck, Zero-Shot Policy Transfer, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qdEBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper derives interpretable transfer guarantees for information-bottleneck policies, assuming known task models and optimal policies. Grid-world and MiniGrid experiments certify 88.6% and 68.4% of near-optimal transfers, respectively, with no false certifications.

Critical strengths
  1. The paper connects theory with experiments, checking both whether the guarantees hold and how tight the bounds are.
  2. It helps explain why transfer fails—for example, whether changed images confuse the agent or the new task requires different actions.
Critical weaknesses

The certificate’s practical use is limited by its reliance on the target-optimal policy(In practice we can already solve the task with the optimal policy and do not need the transfer!). While the paper analyzes different compression levels, it remains unclear what additional insight the framework offers beyond direct performance evaluation. Comparing different transfer methods and showing how the framework explains their relative strengths and failure modes would help clarify its value.

Questions for authors

Could you apply your framework to methods such as domain randomization and representation learning, and show whether it helps explain their performance differences and failure modes?

Reviewer_EZwXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies zero-shot transfer of information bottleneck policies between two MDPs. The error bound is decomposed by source distortion, task disagreement, and decoder sensitivity. The analysis is extended to neural networks with learned representation, with an additional term captures representation shift between source and target observations. The theory is evaluated on tabular grid-worlds and image-based MiniGrid environments.

Critical strengths

The paper is well organized, and the theoretical development is easy to follow.

Although I have concerns about the novelty and practical implications of the results, the presentation is clear and the mathematical guarantees appear technically correct under the stated assumptions.

Critical weaknesses
  1. My main concern is that the theoretical contribution is rather trivial. While the resulting decomposition is technically correct, I do not see new theoretical insight beyond combining these standard decompostioins. Moreover, the theory does not fundamentally depend on the policy having been learned through an information bottleneck -- the guarantee applies to any policy with similar encoder-decoder representation.

  2. the certificate requires substantial oracle information about the target task, including the target-optimal policy, target action-value gaps, and the target discounted occupancy of the transferred policy. This quite limits the scope and significance of the results.

  3. Experimental validation is limited to toy environments and does not really demonstrate the usefulness of the decomposition. The experiment is restricted to 10×10 tabular grid-worlds and 8×8 image-based MiniGrid environments. Even in the neural setting, the underlying MDP is sufficiently small to allow exact computation of optimal policies, action gaps, and state occupancies. Given the theory is rather trivial, I think the paper has very limited signifance if the experiment is also in toy settings.

Questions for authors

see weakness

Reviewer_XqpDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates why a "compressed" policy might fail when transferred from a source MDP to a target. By "compressed" policies, the authors refer to policies with an information bottleneck on the dimensionality of the state representations. They achieve this by (upper) bounding the performance difference between the transferred policy and the optimal policy in the target by terms that disentangle the error due to compression from other factors.

Critical strengths

The problem of identifying why policy transfer might fail, is an interesting and relevant problem. I like how they propose a useful diagnostic through the upper bound that seems to work well in practice.

Critical weaknesses

My main concern with the paper is that evaluating the proposed certificate requires access to the optimal policy in the target, which essentially defeats the purpose of transfer learning.

Questions for authors

Could the authors please clarify if their certificate requires access to the optimal target policy? If yes, what is the utility of their method?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.