acceptodds
Under review as a conference paper at ICLR 2027

Robust-Power-HOOT: Monte Carlo Tree Search for Continuous Stochastic MDPs Under Model Ambiguity

Abstract

We study Monte Carlo Tree Search (MCTS) for continuous-action stochastic MDPs under model ambiguity, where the model used for planning may differ from the execution environment. Existing methods typically address at most two of three challenges: continuous-action exploration, stochastic value aggregation, and distributional robustness. We propose Robust-Power-HOOT, which combines HOO-style continuous-action search, polynomial exploration bonuses, power-mean backups, and conservative distributionally robust Bellman updates. Exact robust optimization at every backup is costly in large search trees; instead, we use closed-form upper bounds on robust penalties to compute conservative lower estimates of robust values. We prove that the root estimate converges at rate , where , to the projected finite-horizon value induced by these lower-bound updates. We decompose the gap to the exact robust objective into finite-horizon truncation, state abstraction, lower-bound approximation, and finite-sample errors, and identify regimes in which the penalty bounds are tight. Across seven continuous-control benchmarks with controlled planning–execution mismatch, the method often degrades less as execution noise increases.

Primary area:
reinforcement learning
Keywords:
Distributionally Robust Reinforcement Learning, Monte Carlo Tree Search, Continuous State-Action Spaces, Robust Planning, Online Planning, Robust Markov Decision Processes

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_47MsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper takes the next step in the evolution starting from the Hierarchical Optimistic Optimization (HOO) to HOOT (application of HOO to trees), POLY-HOOT (HOOT with polynomial bonus term and proof of convergence) and considers the setting of robust optimization where the transition dynamics may be chosen by an adversary. It combines the idea of power-mean backups of Stochastic-Power-HOOT which works in continuous spaces with the robust optimization approach from Robust-Power-UCT which only works in discrete spaces. Thus it fills an important gap in the literature. The paper provides reasonable theoretical guarantees and empirical results in multiple domains under different divergence measures.

Critical strengths

The paper addresses the important problem of solving robust stochastic MDPs and gives an algorithm with theoretical guarantees.

The appendix and the main text provide the full proofs of the claims.

The approach is illustrated in multiple domains with good performance.

Critical weaknesses

Unfortunately, the paper is difficult to read by wider audiences than the experts. Some of these issues can be easily fixed by introducing the main ideas without overwhelming the reader with jargon. For example, the abbreviation of HOO was never explained, and the introduction has loss equations with terms whose meanings were left to the reader's imagination. The introduction should be more basic and should not assume expert knowledge. The equations are not needed at this stage. More interesting would have been an application-oriented motivation. Why is this generalization important in practice apart from its natural mathematical advance and elegance? The authors can see the original HOO paper for a good example.

Most of the paper is filled with claims and proofs which would work well in a 'theory conference' or in a journal. I suggest that the authors focus on the intuition behind the theorems and proofs in the main paper and relegate most proofs to the appendix.

The paper also spends little time in interpreting the empirical results. While they generally seem strong, it is not clear what to make of the results of different divergences, what trends are there if any, and how to explain them. The readers are looking for insights, not just some pretty curves.

Questions for authors
  1. Can you provide a summary of the different claims and the novelty of proof techniques (if any) at a high level before introducing all the formal machinery? is it basically a composition of familiar proof methods and algorithmic tools?

  2. Can you provide a more robust discussion of the experimental results potentially using some ablation studies?

  3. Can your theory support the differences in the empirical performance of the different divergences?

1 comment
AuthorsOpenReview ↗

Our response to Reviewer 47Ms

We want to thank the reviewer for the careful reading and for seeing the gap that this paper fills.

  1. About presentation and motivation. It is our fault: we used acronyms and equations in Section 1 before defining them, and we have fixed this. On the practical motivation, we want to position the story behind the work. We started from two lines of work. Continuous-action tree search (HOO, POLY-HOOT, Stochastic-Power-HOOT) plans as if the simulator were exact, and robust tree search (Robust-Power-UCT) plans for the worst model but needs a finite action set. In control, the actions are torques or steering commands, a grid over them grows as kmk^m (about 10810^8 on Humanoid), and the simulator never matches the real system exactly (Tobin et al., 2017; Peng et al., 2018). So we explicitly ask whether one tree search can handle continuous actions, stochastic rollouts, and model mismatch at once, with a guarantee.

  2. Novelty of the proof (Q1). Yes, we largely compose known tools, and we now open Section 5 with a paragraph that names them. We needed three additional steps. Our closed-form backups estimate a conservative value, and its Bellman operator need not be monotone or a contraction, so we propagate errors only through the exact robust operator (Prop. 4, Cor. 1). Our plug-in penalty depends nonlinearly on child values that are still converging, so we prove its concentration separately (Lemma 1, App. B). And continuous successor states almost never repeat, so we aggregate them and bound the projection bias (Prop. 1).

  3. Ablations (Q2). We now discuss the checks of Appendix H.1 in Section 6. The clearest one is Table H.15. When we replace the closed-form W1W_1 backup by the exact one, the return rises on CartPole (7.4→19.17.4\to19.1) and Pendulum and moves both ways on the MuJoCo tasks. We read this as a sign that our approximation has a cost on some tasks, although this experiment alone cannot show that penalty tightness causes it.

  4. Divergences (Q3). Partly. Prop. 4 says that the approximation error grows with the looseness of the bound, and our diagnostics follow this order. χ2\chi^2 gives the tightest bound (0 to 1% gap, Table H.14) and the smallest worst-case swing over the radius sweep (32%, against 40% for TV and 50% for W1W_1, Table H.17). Our theory cannot predict which divergence gives the highest return, since that depends on how well the ambiguity set matches the real mismatch.

We would be grateful if the reviewer could look again at Sections 1, 5, and 6.

Our references for comments above. Tobin, J. et al. Domain randomization for transferring deep neural networks from simulation to the real world. IROS, 2017. Peng, X. B. et al. Sim-to-real transfer of robotic control with dynamics randomization. ICRA, 2018.

Reviewer_8MCnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper works on a more efficient way of extending Monte Carlo Tree Search (MCTS) to continuous actions with stochastic outcomes under model ambiguity. This work aims to make MCTS more reliable under real world circumstances. To this end, they propose ROBUST-POWER-HOOT which uses Hierarchical Optimisitc Optimization (HOO) for continuous-action search combines with state aggregation, conservative robust bellman backups. The paper introduces meaningful theoretical contributions to provide an upper bound to the cumulative or total error and also tests their algorithm on standard RL benchmarks

Critical strengths
  1. The problem formulation is menaingful
  2. The explicit error computation
  3. Sensible design choice
Critical weaknesses
  1. Theory-Implementation has a mismatch: The theoretical analysis uses fixed representatives for state cells. The practical first-visit implementation instead freezes the first state that reaches a new node, making representatives data-dependent. The paper acknowledges that the corresponding gap is not quantified in the reported logs because cell or merge radii were not recorded.

  2. How is the root value convergence links to action optimality is not clear to me.

  3. The graphs demonstrate huge variance in results. Can the authors provide through light in this huge variance. This draws skepticism towards the error computation.

  4. The TV penalty approximation reaches a reported relative gap of 29% on CartPole. A conservative lower bound can protect against optimistic value estimates, but an overly pessimistic bound can also distort action rankings and suppress useful exploration.

  5. Also a minor comment the colours in the plot specifically the deviations are very difficult to understand due to mixing of various shades together, if the authors can give a tabular coparison of the range, mean(can be seen from plots),standard-deviation or variance, it would be helpful.

Questions for authors

See weaknesses

3 comments
AuthorsOpenReview ↗

Our response to Reviewer 8MCn

We want to thank the reviewer for the constructive review and for recognizing the problem formulation and the explicit error decomposition. Your questions separate three levels of guarantee that we had mixed, and we now separate them in the revision.

  1. First-visit representatives (W1). Our projection bound (Prop. 1) is deterministic, so it holds for any realized representatives. The statistical part stays open: when the representatives are chosen adaptively, the concentration and successor-count conditions (Assumption 5) no longer follow directly from our analysis. If we freeze the representatives in a separate phase and plan with fresh samples, we remove the selection effect, and the theorem applies whenever Assumption 5 holds (for example, with separate child copies per edge).

  2. Root value and action optimality (W2). We agree that root-value convergence alone does not make the returned action optimal, and we now state in Section 5 what we can show. Let g(a)g(a) and f(a)f(a) be the exact robust and conservative root action values in the projected finite-horizon problem, with 0≤g(a)−f(a)≤BH0\le g(a)-f(a)\le B_H. Then

sup⁡ag(a)−g(a^n)≤BH+δn+2en,\sup_a g(a)-g(\widehat a_n)\le B_H+\delta_n+2e_n,

where δn\delta_n measures how well the actions created by HOO cover the action space and ene_n measures our estimation error on them. Our theorem gives no rate for δn\delta_n or ene_n, so we present this bound as conditional. With exact optimization, we can say more. If a policy maximizes our conservative recursion at every node, its robust value lies between V0ub,ψV_0^{\mathrm{ub},\psi} and V0rob,ψV_0^{\mathrm{rob},\psi}, so it loses at most BHB_H in robust value (Appendix A.6). We prove this with the monotonicity of the exact robust operator only, so we never need our surrogate operator to be monotone.

  1. Pessimistic penalties (W4). We agree. Our surrogate can only swap actions whose robust values lie within BHB_H of each other, but BHB_H can be large. We computed the 29% (Table H.14) offline as a relative gap of the penalty at logged CartPole backups, so it measures the bound and says nothing directly about the return. For decisions, Table H.15 gives our closest evidence. With the exact W1W_1 backup, the return rises on CartPole (7.4→19.17.4\to19.1) and Pendulum, which suggests that the approximation costs performance there, although this experiment alone cannot show that penalty tightness causes it. We have not yet run this comparison for TV. The HOO bonuses encourage exploration of our conservative payoff, but they do not correct the approximation gap to the exact robust value.

  2. About the variance. We agree that the returns show large variance, especially on Acrobot. At the highest noise level, the observed returns range from 4 to 78, largely depending on whether the pendulum successfully swings up. Since each point averages only five episodes of 150 steps, the estimated mean is also uncertain. Our theoretical bound, however, concerns the estimation error of the MCTS root value, not the variance of returns during execution. Therefore, the large variance observed in the experiments does not contradict our theoretical result, but the theory also does not guarantee low variance in actual performance.

  3. About the plots. We added one table per task to Appendix H (Tables H.7 to H.13), with the mean and standard error over five seeds for every method and noise level. The standard deviation equals 5\sqrt5 times the standard error.

We would be glad to answer any further question.

Reviewer_8MCnOpenReview ↗

Acknowledgement by reviewer

I thank the authors for their detailed responses

One question remains which is the rate for returned action optimality, but I agree it is implicitly not a part of present work. Hence, it can be a future work.

However, I am not completely convinced with the variance issue. I understand that the theoretical results are not contradicted but when the work deals error computation, the concern raised was not about contradicting theoretical results, but the fact that might be for stochastic continuous control the number of seeds used is insufficient. The ahigh variance might mean high error as well. That was my concern.

The added table helps a lot. Based on the new justifications and clarification. Most of my doubts are satisfied, and I increase my score by 1

AuthorsOpenReview ↗

Our follow-up to reviewer 8MCn

"I am not completely convinced with the variance issue. [...] The high variance might mean high error as well."

Thank you for clarifying. We understand your concern that five seeds may be insufficient to reliably estimate performance differences, particularly on tasks with high return variance. Our previous response focused on the theoretical distinction rather than this statistical limitation.

We agree that reporting standard errors in Tables H.7–H.13 makes the uncertainty visible but does not resolve it. Since each evaluation involves 500 MCTS simulations per planning step across multiple methods and noise levels, we could not complete additional runs within the rebuttal period.

We will increase the number of independent seeds in the revised version and report uncertainty in the differences between methods.

We sincerely appreciate your careful reading, constructive feedback, and willingness to reconsider our work.

Reviewer_1jicOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper presents modification of Power-HOOT method (Dam, 2025) for robust MDP setting. The authors apply the dual representation of robust Bellman equation for (s,a)(s,a)-rectangular MDPs and approximate the corresponding robust penalties to derive computationally feasible backup rule for MCTS, and derive guarantees equivalent to the guarantees of the Power-HOOT. Finally, the authors have demonstrated their algorithm on a set of modified continuous control benchmarks.

Critical strengths

None

Critical weaknesses

The main weakness of the paper is presentation of results and a lack of novelty. In particular, the paper never discuss any difference in the analysis compared to the Power-HOOT paper of Dam et al. 2025 and what makes the theoretical analysis challenging. In particular, I do not understand any challenges in the current analysis compared to the original one, and I would like to understand which steps of the proof actually deviates from the proof of Dam et al. 2025. Additionally, all derivation of dual representation for robust MDPs are well-known in the robust MDP literature (Iyengar 2005), while the properties of lower-approximation of the ground-truth distributional penalties do not seem to be enough to justify the novelty of theoretical results.

The second weakness is an empirical section, which studies the regime which changes the additional Gaussian noise level in standard continuous control benchmarks, and on this set of benchmarks the authors did not show statistically significant improvement of their method on most of the environments. Also, I would heavily recommend the authors instead use already existing robust RL benchmarks such as RRLS (Zouitine et al, 2024) or Robust Gymnasium (Gu et al. 2025).

Zouitine, Adil, et al. "RRLS: Robust reinforcement learning suite." arXiv preprint arXiv:2406.08406 (2024).

Gu, Shangding, et al. "Robust gymnasium: A unified modular benchmark for robust reinforcement learning." International Conference on Learning Representations. Vol. 2025. 2025.

Questions for authors
  • Why the Hopper environment seems to become easier once the level of noise is increasing: the rewards of all methods increases with higher noise, which is counter-intuitive.
  • How the derived bound on a worst-case difference between VrobV^{rob} and VubV^{ub} compares against a bound between non-robust VV and VrobV^{rob} or VubV^{ub}? If a bound of a similar order also holds between solutions of robust and non-robust problems, the current theoretical guarantees would not establish the theoretical improvement upon a trivial non-robust baseline, so a direct comparison and a proper discussion can rule out that possibility.

Also let me articular smaller issues:

  • A lot of quantities are simply not defined in the main text: ΩZ\Omega_Z, SZS_Z, the acronym HOOT is never expanded in the paper;
5 comments
AuthorsOpenReview ↗

Our response to Reviewer 1jic (1/2)

We want to thank the reviewer for the questions. We agree that our original paper did not explain well why robust backups are difficult inside continuous-action MCTS, or what proof steps differ from Dam (2025). We have revised Sections 1 and 5 to address this.

1. About presentation. We agree, and it is our fault. Section 1 now expands HOO and HOOT and defines the terms before using them. We also moved a clearer explanation of the proof differences to the beginning of Section 5.

2. Why continuous-action robust search is computationally challenging. There are two issues. First, we need to search a continuous action space and compute a worst-case expectation at each selected action. A grid with kk values per dimension has kmk^m actions (about 1.3×1081.3\times10^8 for Humanoid with k=3k=3, m=17m=17). HOO avoids this grid, but still needs a robust backup at every selected edge. Second, in stochastic continuous-state MDPs, successor states rarely repeat, so we aggregate them into cells. Child estimates then keep changing, and robust backups must be recomputed.

For example, n=500n=500 simulations of depth H=50H=50 can involve up to nH=25,000nH=25{,}000 selected-edge updates for a planning decision. This counts edge updates, not total runtime. Exact finite-support robust backups are tractable: Robust-Power-UCT reports O(Mlog⁡M)O(M\log M) in its setting for MM successors. We want to avoid repeatedly solving this inner problem; we do not claim that exact robust planning is intractable.

3. What our closed-form backup saves. With MM successor bins and their current values ziz_i, let μ=∑ipizi\mu=\sum_i p_i z_i and ss be their standard deviation. Our TV and χ2\chi^2 lower-bound updates are

\quad\text{and}\quad [\mu-\sqrt{\rho}\,s]^{+}.$$ Both take $O(M)$ operations without an inner optimization solver (Eq. 4.5), the same order as recomputing the nominal mean. Across the selected-edge updates the backup computation is $O(\sum_u M_u)$, at most $O(nHM_{\max})$ if every update has at most $M_{\max}$ bins. This excludes simulation and HOO maintenance. For $W_1$, evaluating a given global Lipschitz bound is similarly simple, although estimating the empirical Lipschitz seminorm may require $O(M^2)$ pair comparisons. So we do not claim an $O(M)$ cost for every version of $W_1$. **4. The $\chi^2$ bound is also exact in a useful regime.** This is a point we did not highlight enough. Appendix A.4 shows that for Pearson $\chi^2$, the candidate worst-case probabilities are $$q_i^*=p_i\left(1-\frac{\sqrt{\rho}}{s}(z_i-\mu)\right).$$ When $s>0$ and all these probabilities are nonnegative, they are feasible and the exact robust expectation equals $\mu-\sqrt{\rho}\\,s$. A sufficient condition is $\rho\le s^2/(\max_i |z_i-\mu|)^2$; for $s=0$ the penalty is zero. We can compute the value and check this condition in $O(M)$ time. We do not claim the $\chi^2$ inequality itself as new. The consequence for our algorithm is that the closed form is not necessarily an approximation: when the relevant reward and transition penalties are tight, the approximation error in Proposition 4 is zero, even at a positive radius. **5. Which proof steps differ from prior work.** We agree that the robust Bellman formulation, divergence inequalities, and HOO exponent recursion are standard. What we needed to add is summarized below. | Part | What we inherit | What we need to establish | | --- | --- | --- | | Continuous-action search | HOO and power-mean recursion | That the conservative payoff satisfies the required assumptions | | Continuous successors | State aggregation for MCTS | The projected robust target and its bias (Prop. 1) | | Edge estimation | Concentration results for previous backups | Joint control of child estimates and plug-in penalties in a clipped backup (Lemma 1, App. B) | | Power-mean backup | Dam (2025) decomposition | Control with a growing, adaptive set of HOO actions (App. F) | | Approximation to robust planning | Standard robust Bellman properties | Propagate excess-penalty error using the exact operator (Prop. 4, Cor. 1) | With fixed penalties, the HOO proof is largely inherited. Plug-in penalties need the extra concentration conditions. Our theorem targets the projected conservative value, not directly the exact robust value.
AuthorsOpenReview ↗

Our response to Reviewer 1jic (2/2)

6. How our bound compares with nominal planning. We agree with the reviewer that a bound depending only on the ambiguity radius can have the same order for the nominal and conservative planners. The difference is clearer when we look at the penalty itself. Let μ\mu be the nominal expectation, σ\sigma the exact robust expectation, Δ=μ−σ\Delta=\mu-\sigma, and Δ‾\overline\Delta our penalty upper bound. Then

\sigma-[\mu-\overline\Delta]_+=\min\\{\overline\Delta,\mu\\}-\Delta.$$ The nominal planner ignores the whole penalty, while ours incurs only the excess of the bound over the exact penalty. Appendix A.6 gives a continuous-action example with $a\in[0,1]$ and $R_a=0.45(1-a)+aB$, where $B$ is Bernoulli with probability $1/2$ and the $\chi^2$ radius is $0.04$. The nominal planner chooses $a=1$ (robust value 0.40); the exact robust planner and our surrogate choose $a=0$ (0.45). We use this example to show that the two objectives can differ. It is not a claim that our finite-budget planner always finds the best action, or that the surrogate always outperforms nominal planning. **7. What we checked experimentally.** We agree that a lower-bound property alone would not show that the approximation is useful, so we added comparisons at both the backup and planning levels. For backup accuracy, Table H.14 compares our closed forms with exact finite-support penalties on logged backups. The reported relative gap for $\chi^2$ is 0--1% across the six tasks included in this diagnostic, while TV reaches 29% on CartPole. This is encouraging for the tightness of $\chi^2$ in these runs, but the table uses an offline support-capped comparison. It is not a uniform population guarantee or a 1% difference in return. For planning performance, Table H.15 compares the closed-form and exact $W_1$ backups in separate planner runs. On CartPole the mean return changes from 7.4 to 19.1, and on Pendulum from 1112.1 to 1194.2; the effect is mixed on the locomotion tasks. This shows that the approximation can matter to the selected actions. It does not isolate penalty tightness as the only cause, and it is not an exact-versus-approximate $\chi^2$ experiment. We also want to be clear that our $O(M)$ statement is an operation-count analysis; we have not reported matched wall-clock speedups for the inner backups. **8. About Hopper.** The reviewer was right, and this question helped us find a real error. Our Hopper environment had two terms that grew with joint-velocity magnitude, so higher execution noise increased every method's return. We removed both terms and reran all methods with five seeds and the same budget. Figure 1 and Table H.7 now show the corrected results: | Method | Noise 0.1 | 0.2 | 0.3 | 0.4 | Mean | | --- | ---: | ---: | ---: | ---: | ---: | | HOOT | 325 | 312 | 302 | 293 | 308 | | POLY-HOOT | 332 | 320 | 304 | 291 | 312 | | PW | 293 | 292 | 292 | 288 | 291 | | Discretized UCT | 342 | 342 | 337 | 330 | 338 | | Stoch.-Power-HOOT ($H=50$) | 320 | 314 | 311 | 306 | 313 | | Stoch.-Power-HOOT ($H=100$) | 310 | 303 | 291 | 284 | 297 | | TV-Robust | 291 | 286 | 283 | 280 | 285 | | $\chi^2$-Robust | 334 | 325 | 315 | 310 | 321 | | $W_1$-Robust | 324 | 309 | 299 | 288 | 305 | The $\chi^2$ variant has the highest sample mean among the continuous-action methods, while discretized UCT is higher. We also agree that it does not degrade more slowly on Hopper: the drop is 7.2% for $\chi^2$ and 4.4% for the matched-depth nominal planner. We apologize for this environment error. We now report the experiments descriptively because five seeds do not establish statistical superiority. **9. About the suggested benchmarks.** We thank the reviewer for pointing us to RRLS and Robust Gymnasium. These would give a better test of structured mismatch than our current Gaussian-noise protocol. Planning with 500 simulations per step in the full simulators is much slower than in our current experiments, so we could not complete those runs during the discussion period. We now state this limitation and see those benchmarks as the next step. We would be grateful if the reviewer could look again at Sections 1 and 5 and Appendices A.4, A.6, and H.
Reviewer_1jicOpenReview ↗

Response to rebuttal

I would like to thank the authors for their rebuttal and their revision.

First of all, I have to admit that I have no idea which questions Answers 2,3,4 and 7 were supposed to answer.

Regarding the difference with the proof, in the provided text in the reviewed paper (which is extremely hard to read since it is a single large paragraph of very dense text) I still cannot understand what substantially different compared to the paper of Dam (2025) from point of view of proof techniques were used: you mentioned few different building blocks, but I cannot understand what were the challenges to do these replacements. For example, you said that one of contributions is to show that your Lemma 1 is satisfied under clipping, but I cannot find the proof of Lemma 1 in your Appendix to even compare it against Lemma 1 of Dam (2025) (and Dam et al. (2024)). Overall, typically clipping do not change substantially many concentration results and can even help them to hold.

Also, the fact that the nominal planning has exactly the same convergence bound (called ambiguity radius) as the proposed robust one makes all the proven theory vacuous in my view. I agree that in practice the difference may occur but I do not see the point to discuss the theory in this case.

Finally, I still believe that the paper should completely rework its empirical section: as authors admit, it currently does not allow to establish a practical advantage of the proposed method due to the size of confidence intervals (as well as a usage of non-standard benchmarks).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: Summary of the revision

We want to thank all three reviewers for their careful feedback. We have uploaded a revised PDF for the reviewers' convenience (we marked all changes in blue), and we answer each reviewer in full in the individual replies.

  1. About presentation. Reviewers 47Ms and 1jic pointed out that we never expanded HOO and HOOT and that we used symbols in Section 1 before defining them. We have fixed this writing error in Section 1. And with the tenth page, we now open Section 5 with how our analysis differs from Stochastic-Power-HOOT and Robust-Power-UCT, and we add a comparison with nominal planning and a bound on the returned action.

  2. About Hopper. Reviewer 1jic asked why every planner gains return as the noise grows on Hopper, and this question led us to an error in our Hopper environment. We apologize for this error. We fixed it, reran all methods, and updated Figure 1 with the new results and their standard errors.

  3. About the experiments. We added one table per task to Appendix H, with the mean and standard error of every method. We agree that five seeds do not show statistical superiority, so we now report the results descriptively in Section 6. Our noise protocol follows the same idea as robust RL benchmarks (we perturb actions and dynamics at test time), and we see RRLS and Robust Gymnasium as the natural next step.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.