acceptodds
Under review as a conference paper at ICLR 2027

GEODE: Geometry-Controlled Decoder Ensembles for Robust Offline Meta-Reinforcement Learning

Abstract

Offline meta-reinforcement learning (OMRL) combines offline RL’s ability to learn from fixed datasets with meta-RL’s capacity to rapidly adapt to unseen tasks, offering a promising framework for safe, interaction-efficient, and generalizable decision-making. However, its performance can deteriorate under context and task-distribution shifts, as errors in inferred task representations are propagated through the predictive models used to learn them. We identify an overlooked source of this brittleness that arises when decoder ensembles produce diverse predictions yet respond redundantly or overly sensitively to changes in the task representation. We introduce GEODE (GEOmetry-controlled Decoder Ensembles), a framework that explicitly shapes the local geometry of predictive supervision. GEODE combines Adaptive Diversity Control to calibrate prediction dispersion, Spectral Functional Diversity to promote complementary local response directions, and Latent-Space Robustness to limit sensitivity to task-inference errors. Predictive reconstruction anchors these objectives to observed rewards and dynamics, yielding task representations that are informative, diverse, and stable without altering the test-time adaptation procedure. Our theoretical analysis provides guarantees for dispersion transfer under context shift, conditioning of local decoder responses, and robustness to task-representation errors. Extensive experiments on six MuJoCo locomotion and Meta-World manipulation benchmarks establish state-of-the-art overall performance across in-distribution and out-of-distribution adaptation, different offline dataset quality regimes, and progressively shifted task distributions under both zero-shot and one-shot evaluation. These results establish predictive geometry as a powerful design principle for robust and generalizable OMRL.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learnin, Offline meta-reinforcement learnin, Distribution Shift, Decoder Ensembles

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XKbyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper approaches the field of context-based offline meta-RL, and particularly the problem of learning representations (contexts) that are robust to distributional shifts, either task shifts or representation shifts. It proposes a method that combines three regularizations that works on top of the predictive models supervision and aims to regularize the geometry of the learned embeddings. The paper provides theoretical analysis of these objectives (error bounds) and experiments in MuJoCo and MetaWorld tasks, in several different contexts: generalization/robustness, sensitivity to offline data, and ablations.

Critical strengths
  • The problem of offline meta-RL is very relevant for Deep RL research, as it addresses learning and online adaptation from offline, fixed datasets - this is a big bottleneck while deploying RL-based adaptive systems.

  • The paper presents extensive evidence to support the method, either theoretical and experimental. Empirical results are usually very positive.

Critical weaknesses
  • The main problem of the paper is clarity and method presentation. Overall, it is a really hard to parse paper due to how it is written.

    • It is really hard to understand the motivation and contribution of the work from the Introduction. First, it is very much mixed with a literature review that list some works alongside their contributions but not really contextualizes and contrasts them with this work.
    • The contribution is described with several technical terms that are so far undefined and failed to provide a clear and intuitive description of the proposed interventions. Terms like “predictive dispersion”, “diversity and stability of local responses”. The names of the proposed interventions are also vague and exhaustively used without a proper clear, intuitive definition: “Adaptive Diversity Control”, “Spectral Functional Diversity”. This makes it really hard to understand and grasp the concepts in the paper and build an intuition of the method, so the scientific communication is really degraded.
    • The paper seems to implicitly assume a specific architecture for offline Meta-RL but fails to communicate it for the reader. The paper describes the general offline meta-rl and shifts in the preliminaries, but does not formally describe the predictive models used in the method and assume this is known by the reader while describing the proposed method. It talks about encoders and decoders, predicting supervision for dynamics and reward, but never describes this architecture in the background section. Not all Meta-RL is necessarily model-based, so it is important to describe correctly what is the starting point of the proposed method.
  • I observed that the presented baselines results are considerably worse than the ones reported in prior published work [1], which compromise the claims here. The proposed method still seems to perform better, but the margins are much less wide.

  • I also noticed that the method introduces a lot of hyperparameters, which are also sensitive across tasks. There is no further evidence of robustness of these hyperparameters, nor any description on the protocol followed for their selection. This is important to understand the practical limitations of the proposed method.

I cannot recommend acceptance without a substantial rewrite of its content, particularly Introduction and Methodology.

EDIT POST-Rebuttal: See my comment below.

[1] Li et. al. Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning. NeurIPS, 2024.

Questions for authors

What was the protocol for tuning the hyperparameters? Did you use the same tasks to select them or a held-out validation task set?

3 comments

Thank you for the detailed feedback on presentation and baselines. We have substantially revised the Introduction and Preliminaries, added a dedicated Related Work section, and show below that GEODE's state-of-the-art result holds even against the numbers published in [1].

Motivation and contribution. The Introduction now follows one line of argument: (1) the inferred task representation z is always somewhat wrong, especially under shift; (2) the decoders that train the encoder pass errors through their Jacobian with respect to z (Eq. 3), and reconstruction accuracy says nothing about that Jacobian's shape; (3) GEODE therefore controls three properties of the ensemble directly. The literature now sits in a Related Work section that contrasts each line of work with ours.

Undefined terms. Each term is now defined in Sec. 2, before the method: prediction dispersion (how much the K decoders disagree on the same input), local response (how a prediction changes when z is nudged slightly), its direction (whether decoders change their predictions in different directions; SFD) and its magnitude (how much a prediction changes for a nudge of fixed size; LR). "Spectral" in SFD refers to the eigenvalue spectrum of the response Gram matrix (Sec. 3.3); "latent-space" in LR refers to robustness to errors in z.

Architecture. Sec. 2 now describes the predictive context-based OMRL pipeline. An encoder maps a context to z, and a model-free offline actor-critic acts on z. The decoder ensemble teaches the encoder what z should capture: K decoders predict reward and state change from (s, a, z), and their errors, propagated through each decoder's Jacobian with respect to z (Eq. 3), form the encoder's learning signal. The quality of z therefore depends directly on the ensemble's geometry, which is exactly what GEODE controls, at no extra cost at test time.

Baselines vs. Li et al. (2024). GEODE remains state of the art even against the numbers published in [1]:

ID returns, mean normalized score (÷ best per environment)
GEODE0.971
UNICORN-SS, published0.921
UNICORN-SUP, published0.847
CSRO, published0.824
FOCAL, published0.815
CORRO, published0.634

Each ID return is divided by the best of the six methods per environment (best ÷ return for HalfCheetah-Vel) and averaged; OOD is excluded because [1] uses a different set of OOD behavior policies. GEODE improves on the best published result on Ant-Dir (+35%) and Reach (+24%), matches it on HalfCheetah-Vel and Walker-Param, and is within 8% on HalfCheetah-Dir and Hopper-Param. No published method is within 20% of the best on every environment; GEODE is.

The margin is largest where task inference is hardest. Trained only on random-policy data (Ant-Dir), GEODE reaches 289 / 291 (ID / OOD), against 81 / 62 for published UNICORN on the same data and even above UNICORN's expert-data result (279 / 262); GEODE keeps 81% / 84% of its expert-data return, against under 30% for published UNICORN and FOCAL. Under task-distribution shift (Fig. 5), GEODE gives the highest zero-shot return in every shifted band of HalfCheetah-Vel, Hopper-Param, and Walker-Param, and remains best or tied after one context episode; in the severe band it reaches 420 one-shot on Walker-Param against 290 for the next best, almost unchanged from its ID level (450).

Where protocols match, our baseline reproduction agrees with [1] (e.g., the data-quality study). Both works build OOD contexts from behavior policies of other tasks, but with different policy sets (ours: 40 source tasks spanning weak, intermediate, and converged behavior), so we compare against [1] on ID returns.

Hyperparameters and tuning protocol. None of GEODE's three loss weights or its perturbation radius was tuned on meta-test performance. Reference values were set once on Ant-Dir from training-time behavior and transferred to every other environment by a fixed rule (App. E.1): radii scale with the latent dimension, the LR weight is rescaled to cancel its dependence on the radius, λ_SFD needs no rescaling because SFD compares normalized responses, and λ_ADC is constant. GEODE thus reaches state-of-the-art performance on six benchmarks with a single reference configuration and no environment-specific tuning. A λ_SFD sweep (Appendix F) shows alignment falling monotonically with λ_SFD (Walker 0.388 → 0.233 → 0.106; Ant-Dir 0.604 → 0.448 → 0.230).

The revised paper is organized around a single argument with every term defined, introduces the architecture explicitly, and establishes GEODE's advantage against the published baselines, with the largest margins where task inference is hardest.

[1] Li et. al. Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning. NeurIPS, 2024.

Dear authors,

Thank you for your efforts on rebuttal. I acknowledge reading your rebuttal and I can see the clarity of the paper improved in the revised draft. Regarding the baselines: I also noticed that the proposed method improves over the baseline published results for specific evaluation setups, but I raise the following concern: this acknowledges that there is a gap between the published results and the ones adopted in this paper, which raise questions whether the other evaluation setups (e.g., OOD, generalization under shifts, etc) are affected due to weaker baselines. In any case, I will assume the best efforts in the experimental setup and acknowledge the fact that sometimes is hard to reproduce results published in papers.

The hyperparameter protocol looks clear to me. It is good to know no tuning specific tuning was needed across tasks, although the sweep suggests the method is quite sensitive to the hyperparameter values.

Overall, I decide to increase my score from 2 to 3.

Dear Reviewer,

Thank you so much for your valubale comments, suggestions and time during the review process. Moreover, we are happy that we could carefully address your comments and are grateful for increase your score from 2 to 3. As you recommended, we will carefully consider addressing your new concern while we are revising the paper.

Reviewer_Sx9COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses performance degradation in context-based offline meta-RL under context shift and task-distribution shift, focusing on the local geometry of the decoder ensemble that provides predictive supervision to the encoder. The authors introduce three losses that shape this geometry: ADC controls the output dispersion of the decoders, SFD controls the direction in which they respond to perturbations of the task representation z, and LR limits the magnitude of that response. The resulting method, GEODE, is intended to provide supervision that is diverse across decoders while remaining insensitive to errors in z. Experiments on six benchmarks under ID and OOD contexts, together with data-quality experiments on Ant-Dir and task-shift experiments in three environments, show advantages over the baselines.

Critical strengths

S1. A New Direction of Controlling the Local Geometry of the Decoder Ensemble

Unlike existing COMRL methods, which focus on the encoder objective or the prediction accuracy of the decoder, this paper is novel in that it takes the local geometry of the decoder ensemble itself, namely the output dispersion and the direction and magnitude of its response to perturbations of z, as a training objective. Each of the three losses is accompanied by a proposition showing that it controls the intended geometric quantity, so the design is not arbitrary but tied to the analysis.  

S2. Broad Evaluation Without Additional Test-Time Procedures

The evaluation spans ID/OOD contexts, offline-data quality, and zero-shot and one-shot task-distribution shift, and GEODE outperforms the baselines overall. In particular, the test-time adaptation procedure is identical to that of the base method, so the method does not rely on additional test-time optimization or context selection as in IDAQ or CERTAIN. In this sense the results are obtained from changes to the training stage alone, and their breadth is the main evidence supporting the claims of the paper.

Critical weaknesses

W1. The Identified Failure Modes Are Not Shown to Actually Occur

The premise of the paper is that a decoder ensemble trained with reconstruction alone responds to perturbations of z in redundant directions or with excessive sensitivity. However, Fig. 1, which supports this premise, is only a conceptual illustration, and although the response Gram matrix in Sec. 3.3 and the perturbation response in Sec. 3.4 are directly computable quantities, no measurements are reported for either the plain ensemble or GEODE. Propositions 1–3 only show that each loss bounds the quantity on which it is defined, and do not address how that quantity relates to the properties of the learned z or to the return. Consequently, while the paper confirms that controlling the geometry improves performance, it does not confirm that the mechanism it proposes is the cause, which leaves the central claim only partially supported.  

W2. The Contribution of the Three Losses Is Not Separated from That of the Ensemble Itself

None of the baselines in Table 1 uses a decoder ensemble, and the ensemble is a component introduced by this paper without being stated as a contribution. The only comparison that isolates the proposed losses is the plain-ensemble row in Table 3, where the gains from adding the three losses are mostly within one standard deviation and performance even decreases on Ant-Dir ID, whereas the plain ensemble alone accounts for most of the gap to the baselines in Table 1. The paper does not discuss this point, nor does it explain the non-monotonic results of the intermediate variants. Moreover, since no K=1 result with the same reconstruction and metric losses is reported, it is difficult to judge whether the improvement over the baselines comes from the ensemble, from the proposed losses, or from implementation differences such as per-environment hyperparameters and the use of CQL.

Questions for authors

Q1. The paper proposes to control redundant response directions and excessive sensitivity to perturbations of z, but it does not show whether these failure modes actually occur. Could the authors measure them for the plain ensemble and for GEODE in Table 3, using the Gram matrix in Sec. 3.3 and the perturbation response in Sec. 3.4, under both ID and OOD contexts? Showing that the plain ensemble exhibits these failure modes and that the proposed losses mitigate them would substantially strengthen the motivation.

Q2. Could the authors report K=1K=1 results trained with the same reconstruction and metric losses? This would clarify how much of the gap to the baselines in Table 1 comes from the ensemble itself. In addition, the standard deviation of some baselines in Table 1 is close to or larger than the mean. Were the baselines tuned per environment to the same extent as GEODE, and were the reported results obtained with the publicly released implementations?

3 comments

Thank you for the detailed review and the concrete questions.

W1 / Q1: Do the failure modes occur, and do the losses mitigate them? We ran the measurement you proposed on trained checkpoints (new Appendix F). On a fixed probe set shared by all methods (40 tasks × 256 transitions), we perturb the inferred task representation along a random unit direction and compute (i) response alignment, the mean |off-diagonal| of the normalized response Gram matrix of Sec. 3.3, batch-stacked as in Eq. 8 (B = 64) and per transition, where 1 means all decoders respond in the same direction; and (ii) the response magnitude ‖J_k v‖/√d_y of Sec. 3.4.

Plain ensemble → GEODEAlignment, Eq. 8 ↓Alignment, per transition ↓Response magnitude
Walker-Param0.388 ± 0.008 → 0.110 ± 0.006 (−72%)0.464 ± 0.003 → 0.360 ± 0.010 (−22%)0.035 → 0.048
Ant-Dir0.604 ± 0.023 → 0.236 ± 0.011 (−61%)0.929 ± 0.013 → 0.362 ± 0.043 (−61%)0.94 → 1.77
HalfCheetah-Dir0.438 ± 0.030 → 0.256 ± 0.025 (−42%)0.556 ± 0.015 → 0.427 ± 0.034 (−23%)0.19 → 0.35

Mean ± std over seeds. Welch tests: Eq. 8 alignment p < 0.001 and per-transition alignment p < 0.05 in every environment. Chance levels: ≈0.02 (Eq. 8) and ≈0.15–0.19 (per transition).

Direction collapse occurs and SFD mitigates it. The plain ensemble's responses are highly aligned (0.39–0.60 vs ≈0.02 by chance). GEODE lowers this by 42–72%, and by 22–61% per transition. A λ_SFD sweep shows a monotonic dose-response (Walker 0.388 → 0.233 → 0.106; Ant-Dir 0.604 → 0.448 → 0.230).

Response magnitude increases moderately and consistently (×1.4 Walker, ×1.9 Ant-Dir, ×1.8 HalfCheetah-Dir). This is expected: SFD acts on directions, not magnitude (Sec. 3.4), and a diversified ensemble requires each member to respond along its own direction. The increase comes at no cost in performance, since the same configuration gives the results of Sec. 4, and Prop. 3 bounds the worst-case prediction change in terms of the LR loss.

Props. 1–3 provide guarantees on each controlled quantity; the measurements above confirm directly that these quantities behave as the theory predicts on trained models (Appendix F, ID contexts).

W2 / Q2: Ensemble vs. losses. We agree, and we now state the ensemble explicitly as part of the design: it is the object whose geometry the losses shape. We add a K=1 model trained with the same reconstruction and metric losses and identical per-environment settings to Table 3, next to the plain ensemble:

ID / OODHC-DirWalker-ParamAnt-Dir
K=11198 ± 19 / 1187 ± 18393 ± 12 / 392 ± 15372 ± 52 / 364 ± 34
Ensemble (K=5)1177 ± 24 / 1162 ± 21385 ± 39 / 389 ± 29394 ± 42 / 366 ± 14
GEODE1200 ± 22 / 1176 ± 27412 ± 34 / 420 ± 29372 ± 31 / 377 ± 11

The three rows share the same backbone and settings, so they isolate our contribution. An ensemble on its own is not enough: without geometry control it does not reliably help and falls below K=1 on HalfCheetah-Dir and Walker-Param. GEODE turns the ensemble into a consistent gain: it improves OOD return over the plain ensemble in all three environments (+14, +31, +11), gives the highest OOD return of the three on Walker-Param (+28 over K=1) and Ant-Dir (+13 over K=1, with the seed spread reduced from ±34 to ±11), and stays within one standard deviation of K=1 on HalfCheetah-Dir. The geometry measurements above show the mechanism: GEODE is the configuration whose decoders no longer collapse onto a shared response direction.

Q2: Baseline tuning. All baselines were run from their official open-source implementations, using the per-environment configurations and training procedures released with each paper and repository, without modification. To ensure a fair comparison, every method uses the same offline datasets, train/test task split, context budget, and evaluation procedure as GEODE. As a check on fidelity, our UNICORN, FOCAL, and CORRO results on the Ant-Dir data-quality study closely match those published by Li et al. (2024); for example, UNICORN with random data reaches 83 / 67 (ID / OOD) in our runs vs 81 / 62 published. The large standard deviations of some baselines come from individual seeds that fail to learn under the released configurations, not from differences in the evaluation protocol.

Presentation. We have revised the Introduction and Sec. 2 and added a Related Work section; please see the general response.

The measurements you proposed are now in Appendix F: direction collapse is present in plain ensembles, GEODE removes it, and the K=1 comparison shows that geometry control is what turns the ensemble into a consistent gain.

Thank you for your careful answer and supplementary experiments. However, there still remains a concern. The contribution of this work comes from the geometric losses, but the performance gain of GEODE compared to w/o ensemble is slight. In Table 3, although GEODE increases performance in Walker-Param (+19 ID, +28 OOD), the performance of GEODE seems to show no big difference (in Ant-Dir, 0 ID, +13 OOD) or to be slightly below w/o ensemble (in HalfCheetah-Dir, +2 ID, -11 OOD), and most of these differences are within one standard deviation. So I maintain my score.

Thank you so much for valubale comments and for your more assessment. We totally understand your concern the performance of our proposed GEODE model. We will surely discuss these results and will include in the revised version of our work. We will provide a more transparent discussion of the quantitative outcomes, and will include the role of the geometric losses beyond the overall performance improvements. To this end, also we will ensure that all of our claims are properly calibrated and totally supported by the experimental results.

We sincerely appreciate your valuable feedback and the time you have spent to evaluating our work. Given the limited time remaining at this stage, we would be very grateful if you could kindly consider our efforts to modify and improve the quality of our work by carefully addressing your concerns and it might help you to reconsider and be positively reflected your current assessment and reflected in your final evaluation. Once again, thank you for your time and consideration.

Reviewer_tK5SOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GEODE, a framework for offline meta-reinforcement learning that improves task representation learning by explicitly controlling the geometry of decoder ensembles. It combines Adaptive Diversity Control,Spectral Functional Diversity, and Latent-Space Robustness to regulate prediction dispersion, response directions, and sensitivity to task-embedding perturbations. Experiments on six continuous-control benchmarks show improved performance under context shifts, varying offline-data quality, and task-distribution shifts.

Critical strengths
  1. Clear motivation and principled design. GEODE distinguishes prediction disagreement from the geometry of decoder responses. Its three objectives address dispersion, response direction, and local sensitivity, with theoretical results characterizing each property under stated assumptions.
  2. Broad evaluation. The paper tests context shift, three offline-data quality levels, and progressively shifted tasks, providing useful evidence beyond standard in-distribution evaluation.
Critical weaknesses
  1. The proposed mechanism lacks direct empirical validation. The paper does not measure whether GEODE actually improves the claimed geometric properties in trained models, or whether those changes explain its downstream gains rather than a general regularization effect.
  2. The combined objectives do not consistently beat simpler variants. In Table 3, full GEODE has lower Ant-Dir ID return than the plain ensemble and both intermediate variants; on Walker-Param OOD, ADC alone is slightly higher. Given the reported variation, the incremental benefit of SFD and LR remains unclear. A sensitivity analysis would also help assess dependence on their weights and perturbation radii.
Questions for authors

See the weaknesses

1 comment

Thank you for the positive assessment and the precise suggestions.

W1: Direct validation of the mechanism. We now measure the geometry directly (new Appendix F) on a fixed probe set shared by all methods (40 tasks × 256 transitions): response alignment, the mean |off-diagonal| of the normalized response Gram matrix G of Eq. 8 (1 = all decoders respond in the same direction), batch-stacked as in Eq. 8 (B = 64) and per transition.

Plain ensemble → GEODEAlignment, Eq. 8 ↓Alignment, per transition ↓
Walker-Param0.388 ± 0.008 → 0.110 ± 0.006 (−72%)0.464 ± 0.003 → 0.360 ± 0.010 (−22%)
Ant-Dir0.604 ± 0.023 → 0.236 ± 0.011 (−61%)0.929 ± 0.013 → 0.362 ± 0.043 (−61%)
HalfCheetah-Dir0.438 ± 0.030 → 0.256 ± 0.025 (−42%)0.556 ± 0.015 → 0.427 ± 0.034 (−23%)

Welch tests: p < 0.001 (Eq. 8) and p < 0.05 (per transition) in every environment. Chance levels ≈0.02 and ≈0.15–0.19.

The failure mode of Fig. 1(a) is clearly present: plain-ensemble members respond to changes in z in nearly the same direction, even though their outputs differ. GEODE removes most of this alignment, across the batch and per transition. Varying λ_SFD with everything else fixed lowers alignment monotonically (Walker 0.388 → 0.233 → 0.106; Ant-Dir 0.604 → 0.448 → 0.230 for λ_SFD = 0, 0.1, 1.0), so the effect is caused by SFD rather than generic regularization.

On whether the geometry explains the gains: adding decoders without geometry control does not help and can even hurt, as the plain ensemble falls below K=1 on HalfCheetah-Dir and Walker-Param. GEODE turns the ensemble into a consistent gain: it improves OOD return over the plain ensemble in all three environments (+14, +31, +11), most on Walker-Param (+19 / +28 ID / OOD over K=1), and on Ant-Dir it reaches the best OOD return of all variants with the joint-smallest seed spread (377 ± 11 vs 364 ± 34 for K=1). Its decoders no longer collapse onto a shared response direction (Appendix F). The geometric objectives are what make the ensemble worth having: they turn a component that is neutral or harmful on its own into one that improves robustness under context shift.

W2: Combined objectives vs. simpler variants. Several individual increments in Table 3 are within one standard deviation; Table 3 now also includes K=1. GEODE's goal, however, is robustness across settings rather than the largest gain on one benchmark, and this is where the full objective pays off:

  • Ablation: full GEODE is best in three of the six columns of Table 3 (HalfCheetah-Dir ID, Walker-Param ID, Ant-Dir OOD), more than any other variant, and within one standard deviation of the best in the rest.
  • Across benchmarks: prior methods excel on some environments but collapse on others. Against the numbers published by Li et al. (2024), UNICORN is best on four of six environments yet drops to 72% of the best on Ant-Dir and 81% on Reach; no published method stays within 20% of the best everywhere. GEODE does (worst case 92%), with the highest mean normalized score (0.971 vs 0.921).
  • Data quality: the gap widens under harder conditions. Trained on random-policy data, GEODE retains 81% / 84% of its expert-data return on Ant-Dir (under 30% for published UNICORN and FOCAL) and outperforms published UNICORN trained on expert data (289 / 291 vs 279 / 262).
  • Task-distribution shift (Fig. 5): GEODE gives the highest zero-shot return in every shifted band of all three benchmarks, before seeing any task-specific context, and remains best or tied after one context episode. In the severe band it reaches 430 zero-shot and 420 one-shot on Walker-Param (next best 345 and 290) and 390 zero-shot on Hopper-Param (next best 210), and its one-shot Walker-Param return stays nearly flat from ID to severe shift (450 → 420).

GEODE is thus the only method compared that delivers state-of-the-art performance consistently across benchmarks, data quality, context shift, and task-distribution shift, in both zero-shot and one-shot evaluation, without collapsing on any of them.

Sensitivity analysis. The λ_SFD sweep is in Appendix F. Radii and LR weights for other environments are not tuned but derived from the Ant-Dir reference by the scaling rule in App. E.1, so GEODE reaches these results with a single reference configuration and no environment-specific tuning.

Your suggestions made GEODE's mechanism directly measurable: the failure mode it targets is present in plain ensembles, GEODE removes it with a clear dose-response, and the full method delivers the most consistent performance across benchmarks.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers for their constructive feedback. Reviewers recognized controlling the local geometry of decoder ensembles as a new direction, the breadth of the evaluation, and that GEODE requires no additional test-time procedure. The revision strengthens the paper on every point raised:

  1. GEODE demonstrably changes the geometry it targets (new Appendix F). Measured directly on trained checkpoints, the failure mode GEODE addresses is real: in a plain ensemble the decoders' responses to a change in z are strongly aligned (|cos| = 0.39–0.60, against ≈0.02 for random directions). GEODE reduces this alignment by 42–72% in all three ablation environments (p < 0.001), and by 22–61% for individual transitions. A sweep over λ_SFD gives a monotonic dose-response, establishing SFD as the cause.
  2. Geometry control is what makes the ensemble useful. We add a single-decoder model (K=1) with identical settings to Table 3. A plain ensemble does not reliably improve on it and is worse on two of three environments; GEODE turns the ensemble into a consistent gain, improving OOD return over the plain ensemble in every environment.
  3. Consistent state of the art. Against the numbers published by Li et al. (2024) [1], GEODE achieves the highest mean normalized score (0.971 vs 0.921 for the strongest baseline) and is the only method within 20% of the best on every benchmark. With random-policy data, it retains 81% / 84% (ID / OOD) of its expert-data return, against under 30% for prior methods. Under task-distribution shift (Fig. 5), GEODE gives the highest zero-shot return in every shifted band of HalfCheetah-Vel, Hopper-Param, and Walker-Param, and remains best or tied after one context episode.
  4. Clearer presentation. The Introduction now runs problem → gap → idea → contributions, a dedicated Related Work section positions GEODE against each line of work, and Sec. 2 introduces the architecture and defines every geometric term in plain language.

We also welcome any further suggestions during the discussion period and will incorporate them into the revision before the deadline.

[1] Li et. al. Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning. NeurIPS, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.