acceptodds
Under review as a conference paper at ICLR 2027

Distractor-Robust Reinforcement Learning via Variational Bisimulation

Abstract

Model-based reinforcement learning (MBRL) promises data efficiency and generalization, but typical reconstruction-based objectives encourage models to waste representational capacity on task-irrelevant distractors. We introduce VIBES (Variational Inference for Bisimulation-based Encoded States), a new objective that replaces pixel reconstruction with an adversarial term, which enforces that latent states suffice to predict both rewards and the next latent state. We show theoretically that, under mild assumptions, global optima of this objective correspond to encoders that induce bisimulation relations, ensuring that latent states capture task-relevant information while discarding irrelevant variation. Our method serves as a drop-in replacement for Dreamer’s model-learning component, achieves state-of-the-art performance on the Distracting Control Suite, and extends to visually complex manipulation in Realistic ManiSkill. Unlike prior approaches, it does not rely on image-specific augmentations and applies equally well to high-dimensional vector-state tasks, demonstrated on a 100-link swimmer robot. Finally, latent-space analyses (UMAP embeddings and nearest-neighbor probes) confirm that the learned representations are sensitive to task-relevant structure while invariant to distractors.

Primary area:
reinforcement learning
Keywords:
state abstraction, variational inference, bisimulation relations, model-based reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Ja8mOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

disclaimer: I also reviewed a previous version of this work. This version adds stronger baselines, manipulation experiments and clear details. I might be slightly biased.

This paper proposes VIBES, which replaces pixel reconstruction in model-based reinforcement learning with an adversarial objective that encourages representations to preserve reward and dynamics information while ignoring distractors. The method is evaluated on the Distracting Control Suite, Realistic ManiSkill and a high-dimensional swimmer task. The revision expands the baseline comparisons and manipulation experiments, and provides more detailed implementation and representation analyses.

Critical strengths

(S1: Well-motivated approach) Avoiding reconstruction of irrelevant observations is a clear motivation, and the adversarial formulation provides an interesting approach to learning bisimulation-based representations. The paper is easy to follow.

(S2: Broader empirical evaluation) The additional HRSSM, RePo and DreamerV3 comparisons strengthen the evaluation. The manipulation experiments broaden the evidence, while the swimmer results demonstrate applicability beyond image observations.

Critical weaknesses

(W1. Representation analysis does not isolate the proposed component) Return wise, Reacher-easy achieves 925 without the adversary, compared with 930 for proposed method VIBES. The representation analysis in Figure 4 nevertheless focuses on this task without comparing against the non-adversarial version. My question is therefore, which properties of the learned representation actually depend on the adversarial term? This is probably the main weakness in understanding the proposed method.

(W2 types of distraction) (not critical, this was one of my previous suggestions too). Background, camera and colour distractions are evaluated together. Understanding which distractions is the bisimulation especially good against would be a good additional contribution.

(W3. Adversarial contribution on ManiSkill) (not critical) The authors reasonably explain why the adversarial term contributes little on some DCS tasks. However, I think ManiSkill tasks can be configured to at least test the fully observable MDP hypothesis. A non-adversarial ablation would clarify how much the adversarial term contributes to ManiSkill.

Questions for authors

I would consider raising my score further if the questions are suitable responded. Q is for questions, C is for comments.

(Q1: Addresses W1) How do the representations differ with and without the adversary on Reacher-easy and on a task where the performance gain is larger? That is, could the authors compare VIBES and no adversarial objective on a common evaluation set, using Reacher-easy and one task with a larger return gain?

(Q2: Addresses W2) Which types of distraction benefit most from the adversarial term? Can separate background, camera and colour evaluations support the proposed explanation?

(Q3: Addresses W3) Does the adversarial term improve performance on the new manipulation tasks when compared with the same model trained without it?

(C1.) Please include DCS learning curves alongside the final returns in Table 1. These would help distinguish improvements in learning speed from improvements in final performance.

(C2) Related to C1, I think Figure 3 was never cross-referenced in the manuscript. Adding more learning curves would help.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Ja8m

We thank the reviewer for the constructive review. In response, we have made several additions to the paper: we ran the no-adversary ablation on Realistic ManiSkill, added learning curves on the Distracting Control Suite alongside the final returns in Table 1, and are conducting a representation-level comparison with the no-adversary ablation, which will appear in the final version.

Our additions

1. No-adversary ablation on Realistic ManiSkill (addresses W3 and Q3). We ran the no-adversary ablation on both tasks (6 seeds) and added it to Table 2:

Lift CubeTurn Faucet
VIBES561.6 ± 69.0757.2 ± 76.3
No Adv. Obj.509.3 ± 81.5364.5 ± 261.3

On Lift Cube, the difference is not significant (Welch’s t-test, p = 0.36). This is consistent with the hypothesis in Section 4.1: the adversarial term matters least when the task-relevant state is fully observable, and Realistic ManiSkill’s static backgrounds obscure the agent and objects far less than the moving distractors in DCS. On Turn Faucet, VIBES is significantly better (p = 0.03), and the difference lies mainly in consistency across seeds. VIBES seeds are tightly clustered, whereas no-adversary seeds vary widely: two reach returns comparable to VIBES (673 and 817), two reach intermediate returns (238 and 389), and two never learn the task (20.4 and 50.6, close to the random-action return of 35.7). This is compatible with the same hypothesis, which concerns only where training converges: even when the adversarial term contributes nothing at the optimum (as we hypothesize for fully observable environments), it supplies useful gradient early in training, when the encoder is still missing information (consistent with the large adversary–dynamics likelihood gap early in training, shown in the new Appendix C.7). Here, removing the adversary makes learning substantially less reliable across seeds.

2. Learning curves on the Distracting Control Suite (addresses C1 and C2). We have added learning curves for the DCS tasks in Appendix C.8. Figure 3 is now also cross-referenced in Section 4.3.

3. Planned representation analysis (addresses W1 and Q1). To further investigate the effect of the adversarial term, we plan a latent-space analysis on two Distracting Control Suite tasks: Reacher-easy, where the no-adversary ablation nearly matches VIBES’s final return (925 vs. 930), and Ball-in-cup-catch, where the two differ substantially (593 vs. 844). For each task, we will compare the latent spaces of both models using the analyses of Section 4.4, to isolate the effect of the adversary on the learned state representation. This requires more compute time than the discussion period allows, and the results will be included in the final version.

Thank you for the additional experiments on ManiSkill. I am satisfied with the responses. While the authors could not fully address W1, I think their commitment to addressing it is sufficient. I'm happy to maintain my score.

Reviewer_pPXTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VIBES, an objective for distractor-robust model-based reinforcement learning that does not require pixel reconstruction. The method combines reward prediction with latent dynamics sufficiency: a latent dynamics model predicts the next latent state from a compressed state representation, while an adversarial predictor conditions on the full history of observations and actions.

Critical strengths
  1. The paper addresses an important problem with a clear methodological motivation. Pixel reconstruction encourages world models to encode details that are irrelevant to control.
  2. The DCS results are strong and supported by informative ablations.
  3. The representation analyses and vector-input experiments provide additional supporting evidence.
Critical weaknesses
  1. The proof of the main theorem requires further justification. Theorem 3.2 uses the non-negativity of the two KL terms to conclude that both vanish at a global optimum, but does not establish that they can be simultaneously zero. The applicability of the theoretical guarantee to the Gaussian noise used in the encoder also requires clarification.
  2. The theoretical justification for discarding distractors is insufficient. A representation satisfying the bisimulation conditions may still retain irrelevant background information. Preserving task-relevant information does not necessarily imply excluding distractors; additional justification is needed, or the corresponding claims should be qualified.
  3. The paper should more directly compare its objective with the related objectives in [1] and [2]. The existing ablations demonstrate that the adversarial term is beneficial, but do not clearly establish its distinct contribution relative to prior methods. [1] Bridging State and History Representations: Understanding Self-Predictive RL, ICLR 2024. [2] RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictability, NeurIPS 2023.
Questions for authors
  1. Under what conditions can both KL terms in Theorem 3.2 vanish simultaneously? Are these conditions satisfied by the Gaussian encoder with a fixed standard deviation of 0.01?
  2. Does the “state” used to define the bisimulation relation refer to the encoder mean, the encoding distribution, or the full representation including the GRU hidden state? Please clarify how this definition corresponds to the implementation.
  3. How does the objective encourage the representation to discard predictable but task-irrelevant background information? Is this property guaranteed theoretically or primarily supported by the experimental results?
  4. How does VIBES differ from the approaches of [1] and [2] in terms of its objective, the information used by its predictors, and its theoretical properties?
3 comments
AuthorsOpenReview ↗

Response to Reviewer pPXT (1/2)

We thank the reviewer for pointing out real ambiguities in our manuscript: the missing realizability assumption in Theorem 3.2, the gap between bisimulation and discarding distractors, and how our theoretical state corresponds to the implementation. Addressing them has made the theory more precise and our claims more carefully qualified; the changes are in the revised Sections 3.1–3.2 and Appendix B.

Weakness 1: Theorem 3.2, the joint-realizability assumption, and its applicability to our noisy encoder (also answers Q1)

As the reviewer points out, concluding that both KL terms vanish at a global maximum requires that some encoder can make them vanish simultaneously. The revised Theorem 3.2 now states this explicitly as a joint-realizability assumption: an encoder and model exist that make both the reward and dynamics predictions exact (equation 5). Under this assumption, every global maximizer satisfies both conditions and, with full data coverage and an identifiable encoder family, induces a bisimulation (new Lemma 3.1).

Our implemented encoder, with fixed standard deviation 0.01, does not satisfy this assumption, and we now state this explicitly (new Proposition B.3): with fixed Gaussian noise, every latent value is possible under every history, so exact sufficiency fails except in trivial tasks. Here, a trivial task is one in which the history carries no information about the reward or the next latent beyond the previous action, for example one where the reward is constant. Two histories with different rewards can only be confused when the encoder produces nearly the same latent for both, and when their encoder means are far apart relative to the noise scale (σ = 0.01), this is very unlikely, so the encoder can come close to sufficiency even though it cannot reach it exactly (Appendix B.4).

Theorem 3.3 provides a precise mathematical decomposition that accommodates our noisy encoder. It now shows, for any encoder and without assuming realizability, that our objective decomposes as L = C − Δ(ϕ) − M(θ, ϕ), where C is a constant (the best reward prediction achievable from the full history), Δ(ϕ) ≥ 0 measures how much predictive information about the next latent and reward the history carries beyond the latent (the encoder’s sufficiency violation), and M(θ, ϕ) ≥ 0 is the misfit of the learned models. For our noisy encoder, Δ(ϕ) is always strictly positive, but maximizing the objective drives the encoder toward the smallest sufficiency violation achievable within its class, which, as discussed above, is small when encoder means are well separated relative to the noise. Theorem 3.3 is therefore the guarantee that governs our implementation, while Theorem 3.2 describes the idealized limit in which the violation reaches zero.

Weakness 2: Bisimulation does not by itself imply discarding distractors (also answers Q3).

We have qualified the corresponding claims in the revised paper (Sections 3.1–3.2, Limitations). Our objective (equation 2) on its own has no incentive either to encode or to discard predictable distractors. As the amount of predictable distracting information encoded in the latent state increases, the dynamics term log⁡p^θ\log \hat p_{\theta} and the adversary term log⁡p^ψ∗\log \hat p_{\psi^{\ast}} both become better at predicting the next latent state by exactly the same amount, and the two cancel. However, our noisy encoder introduces an information bottleneck in the latent state, limiting its information content to at most some BB bits. Our complete method therefore prefers state representations that encode bisimulation relations while also containing fewer than BB bits. This information bottleneck directly incentivizes distracting information to be dropped, assuming the full state cannot be encoded in fewer than BB bits.

Put another way, encoding distractors does not improve the dynamics sufficiency term in the objective, but when the information bottleneck binds, encoding distractors actively worsens the reward prediction term, resulting in an encoder that prefers to discard distractors.

The objective’s indifference to whether distractors are encoded follows from the theory; the bottleneck’s effect relies on the stated assumption and is not formalized, so the property is primarily supported empirically.

Empirical support. Realistic ManiSkill provides a direct test, since its static backgrounds are predictable distractors. Only 8.0 ± 3.3% (Lift Cube) and 5.6 ± 2.4% (Turn Faucet) of each frame’s nearest latent neighbors share its background scene, close to the 1.1% expected of a background-independent encoder (Section 4.4).

AuthorsOpenReview ↗

Response to Reviewer pPXT (2/2)

Weakness 3: Comparison with self-predictive RL [1] and RePo [2] (also answers Q4). We compare the three methods along the requested axes and then summarize the practical consequence.

Objective. RePo maximizes reward log-likelihood subject to a budget on the KL between a posterior that sees the next observation and a prior that does not. This encourages reward-predictive latents and discourages encoding information the prior cannot anticipate. Self-predictive RL trains an actor-critic loss together with a self-prediction loss: from the current latent and action, it predicts the next latent, and the prediction is compared against the next step’s encoding produced by a slowly updated (EMA) copy of the encoder, which prevents collapse. This encourages latents that are useful for control and whose future can be predicted from their present. VIBES maximizes reward and dynamics log-likelihood minus the log-likelihood of an adversary that predicts the same next latent from the full observation history, excluding the current observation. This encourages latents that lose no predictive information about either the next latent or the reward, relative to the raw history.

Information used by the predictors. In RePo, the prior conditions on the current latent and action, and the posterior additionally conditions on the next observation. In self-predictive RL, the predictor conditions on the current latent and action, and its target is the next step’s encoding from an EMA copy of the encoder. In VIBES, the dynamics model conditions on the previous latent and action, while the adversary conditions on the raw history of observations and actions; both predict the same next latent.

Theoretical properties. RePo derives its objective as a variational information-bottleneck bound and states no formal guarantees beyond these derivations. Self-predictive RL shows that self-prediction together with Q∗Q^{\ast}-sufficiency implies reward sufficiency, and that stop-gradient targets avoid collapse (under linear assumptions). VIBES’s objective decomposes, for any encoder, into a constant minus the encoder’s sufficiency violation and the models’ misfit (Theorem 3.3); when exact sufficiency is achievable, its optimum induces a bisimulation (Theorem 3.2). See our response to W1 for details.

Consequence for distractors. RePo and self-predictive RL remove distractors by penalizing unpredictability; RePo states this assumption explicitly (“spurious variations are not predictable”). A distractor that is predictable but reward-irrelevant therefore incurs no cost under either objective. This leads to a fundamental weakness in both algorithms, which incentivizes the encoding of predictable distractors into the state representation. In contrast, the VIBES objective has no incentive to encode distractors, regardless of their predictability, and when paired with an information bottleneck, actively discourages their encoding (see our response to W2). Realistic ManiSkill, whose static backgrounds are predictable distractors, is consistent with this distinction: VIBES significantly outperforms RePo on both tasks (561.6 vs. 83 on Lift Cube and 757.2 vs. 92 on Turn Faucet; Table 2).

Question 2: Which “state” defines the bisimulation relation. The relation is defined by the encoding distribution: two histories are grouped if and only if the encoder maps them to the same latent distribution, which for our fixed-variance Gaussian encoder is equivalent to grouping them by the encoder mean. In our implementation, the dynamics model, actor and critic use an augmented state (ht,s~t)(h_{t}, \tilde s_{t}), where hth_{t} is the dynamics model’s deterministic recurrent state, computed from past latents and actions. All results in Section 3.2 hold for this augmented state; the formal proof is given in new Appendix B.5.

[1] Bridging State and History Representations: Understanding Self-Predictive RL, ICLR 2024. [2] RePo: Resilient Model-Based Reinforcement Learning by Regularizing Posterior Predictability, NeurIPS 2023.

Thank you for the detailed response and clarifications, which have largely addressed my main concerns. I will maintain my overall assessment.

Reviewer_dQgeOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes VIBES, a model-learning objective for distractor-robust model-based reinforcement learning. Instead of reconstructing observations, VIBES combines reward prediction with an adversarial dynamics-sufficiency term: a latent dynamics model conditioned on the previous latent/action should predict the next latent as well as an adversary conditioned on the full observation-action history. The authors claim that global optima induce bisimulation relations over histories. VIBES is evaluated on the Distracting Control Suite, two Realistic ManiSkill tasks, a custom 100-link vector-state swimmer, and latent-space probes. Empirically, VIBES obtains a DCS aggregate score of (701\pm29), compared with (599\pm58) for HRSSM and (277\pm10) for DreamerV3. The central theoretical claims, however, contain substantial gaps.

Critical strengths
  1. Important problem and a compelling empirical direction. Removing pixel reconstruction is well motivated for distractor-heavy control. On DCS, VIBES achieves the best aggregate mean and the best per-task mean on four of six tasks.

  2. Useful ablations and broader-than-usual evaluation. Within the proposed architecture, VIBES improves over both reconstruction and NoAdv objectives. The ManiSkill nearest-neighbor analysis also provides evidence that its stochastic latent encodes less background information: same-scene neighbor rates are (8.0\pm3.3%) and (5.6\pm2.4%), versus approximately (41%) for DreamerV3-12M.

Critical weaknesses
  1. The conclusion of Theorem 3.2 does not follow from its stated assumptions. In Appendix B.2 (p. 15, Equations 770–774), the objective is written as a constant minus two nonnegative KL terms, after which the proof concludes that any global maximum makes both terms zero. This conclusion requires joint realizability: there must exist an encoder for which reward and dynamics sufficiency are simultaneously achievable. The theorem assumes only expressive reward and dynamics models, which cannot make an information-losing encoder sufficient. The practical encoder is a unit-norm mean plus fixed Gaussian noise, and the reward model also has a minimum standard deviation; exact equality with arbitrary history-conditioned distributions is therefore not guaranteed. Consequently, the proof does not establish the claimed exact-bisimulation guarantee for either the stated model class or the implemented parameterization.

  2. Appendix B.3 does not establish the gradient identity used to justify the alternating update. From Equation 37 to Equation 38 (p. 16), the displayed differentiation drops the dependence of the density inside the expectation on ϕ\phi; the outer distribution over past latents also depends on ϕ\phi. Fixed variance of the per-observation Gaussian encoder does not remove these dependencies: after conditioning on the past and marginalizing the current observation, pϕ(st∣history)p_\phi(s_t\mid history) is generally a mixture whose entropy can change with the encoder means. If the authors intend to differentiate only an explicit occurrence of ϕ\phi while holding the sampling distribution fixed, they need to define that partial derivative and show why it corresponds to the gradient used by the implementation. As written, Equation 41 does not establish that the relevant adversarial gradient vanishes.

Questions for authors
  1. Can the authors provide a corrected Theorem 3.2 that explicitly assumes existence of an encoder satisfying both zero-KL conditions? Under what conditions can the implemented fixed-variance Gaussian encoder satisfy exact reward sufficiency?

  2. Which dependence on ϕ\phi is differentiated in Equations 33–41: the total derivative, a score-function derivative with the sampling distribution fixed, or the pathwise derivative through encoder-generated latents? In the implementation, are gradients propagated through the fixed adversary's outputs with respect to encoder-generated inputs and targets, or is that path stopped?

  3. Can the authors report direct diagnostics of the bilevel approximation, such as the adversary-versus-dynamics likelihood gap and sensitivity to adversary updates/capacity?

  4. Table 2 does not establish that VIBES and DreamerV3-12M are statistically indistinguishable: overlap of their marginal confidence intervals is neither a difference test nor an equivalence test. What estimand and decision criterion support this wording? Otherwise, the claim should be limited to the reported point estimates and intervals.

3 comments
AuthorsOpenReview ↗

Response to Reviewer dQge (1/2)

We thank the reviewer for reading our proofs closely and identifying two genuine gaps: the missing joint-realizability assumption in Theorem 3.2 and the gradient step in Appendix B.3. Both are now fixed in the revised Section 3.2 and Appendix B.

Weakness 1: Theorem 3.2, the joint-realizability assumption, and its applicability to our noisy encoder (also answers Q1)

As the reviewer points out, concluding that both KL terms vanish at a global maximum requires that some encoder can make them vanish simultaneously. The revised Theorem 3.2 now states this explicitly as a joint-realizability assumption: an encoder and model exist that make both the reward and dynamics predictions exact (equation 5). Under this assumption, every global maximizer satisfies both conditions and, with full data coverage and an identifiable encoder family, induces a bisimulation (new Lemma 3.1).

Our implemented encoder, with fixed standard deviation 0.01, does not satisfy this assumption, and we now state this explicitly (new Proposition B.3): with fixed Gaussian noise, every latent value is possible under every history, so exact sufficiency fails except in trivial tasks. Here, a trivial task is one in which the history carries no information about the reward or the next latent beyond the previous action, for example one where the reward is constant. Two histories with different rewards can only be confused when the encoder produces nearly the same latent for both, and when their encoder means are far apart relative to the noise scale (σ = 0.01), this is very unlikely, so the encoder can come close to sufficiency even though it cannot reach it exactly (Appendix B.4).

Theorem 3.3 provides a precise mathematical decomposition that accommodates our noisy encoder. It now shows, for any encoder and without assuming realizability, that our objective decomposes as L = C − Δ(ϕ) − M(θ, ϕ), where C is a constant (the best reward prediction achievable from the full history), Δ(ϕ) ≥ 0 measures how much predictive information about the next latent and reward the history carries beyond the latent (the encoder’s sufficiency violation), and M(θ, ϕ) ≥ 0 is the misfit of the learned models. For our noisy encoder, Δ(ϕ) is always strictly positive, but maximizing the objective drives the encoder toward the smallest sufficiency violation achievable within its class, which, as discussed above, is small when encoder means are well separated relative to the noise. Theorem 3.3 is therefore the guarantee that governs our implementation, while Theorem 3.2 describes the idealized limit in which the violation reaches zero.

Weakness 2: Gradient identity in Appendix B.3 (also answers Q2).

The reviewer is right that the step from Equation 37 to Equation 38 dropped the dependence of the sampling distribution on ϕ, and we have replaced this derivation with a new argument (Proposition B.2, Appendix B.3). In short, the gradient we use is the pathwise derivative through encoder-generated latents. Because the encoder is reparameterized, s~t=μϕ(o≤t,a<t)+σeϵt\tilde s_{t} = \mu_{\phi}(o_{\le t}, a_{<t}) + \sigma_{e} \epsilon_{t}, every dependence of the adversarial term on ϕ, including through the past latents the adversary conditions on and the latent it predicts, enters through these samples, and no score-function term arises. Proposition B.2 then uses an envelope-theorem argument to show that this pathwise derivative, taken with the adversary’s parameters held fixed, equals the total derivative of the objective: because the optimal adversary already maximizes its own objective, its dependence on ϕ contributes nothing to the gradient, so it need not be differentiated. In the implementation, the adversary’s parameters are frozen during the model update, but gradients propagate through its inputs and targets to the encoder; this path is not stopped (Algorithm 1, line 16; Appendix C.1).

AuthorsOpenReview ↗

Response to Reviewer dQge (2/2)

Question 3: Diagnostics of the bilevel approximation.

Over training, the likelihood gap between the adversary and the dynamics model shrinks by roughly 15× while remaining positive throughout. Figure 6 in the revised paper (Appendix C.7) reports this gap, Δ^\hat{\Delta} = dynamics NLL − adversary NLL, on Cheetah-run (5 seeds).

The left panel shows the gap: the mean and its 95% CI stay positive for the entire run, so the adversary always retains a predictive advantage and the adversarial term supplies gradient throughout training. The right panel shows the two losses separately: both decrease, so the narrowing reflects the dynamics model catching up to the adversary rather than the adversary degrading.

Regarding sensitivity, we use one adversary update per model update, with an adversary learning rate 3× the encoder’s (3×10⁻⁴ vs. 10⁻⁴) so that it tracks its optimum as the encoder changes. The adversary is intentionally larger than the dynamics model (1.87M vs. 0.30M parameters) because it encodes raw frames and the full history. These settings were chosen in preliminary experiments against a small set of alternatives; a systematic sweep over update ratio and adversary capacity exceeds the rebuttal timeline and will be included in the final version.

Question 4: Statistical significance of reported results. We agree: CI overlap is neither a difference test nor an equivalence test. We now compare methods with a two-sided Welch’s t-test (α = 0.05), which tests for a difference in mean return without assuming equal variances. Changes to the paper:

  • Underlines in Tables 1 and 2 now mark results not significantly different from the best entry under this test.
  • A new Appendix D reports p-values comparing VIBES with every method.
  • All claims of “statistically indistinguishable” performance have been removed.

On Realistic ManiSkill, VIBES shows no significant difference from DreamerV3-12M (p = 0.25 and 0.11) while using fewer than one-third as many learned parameters; we make no equivalence claim.

Thank you for carefully addressing the theoretical issues raised in my review. The revised assumptions and derivations resolve my main concerns. I hope the authors will incorporate these corrections into the final manuscript. I have changed my rating from 2 to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.