acceptodds
Under review as a conference paper at ICLR 2027

Deep Reinforcement Learning with Incomplete Symbolic Knowledge in Parametrized Action Spaces

Abstract

Specifying a complete symbolic model of a decision-making domain is difficult, tedious, and error-prone. However, even incomplete domain knowledge can constrain decision spaces without providing a complete domain model. In this paper, we study how Deep Reinforcement Learning can use such incomplete symbolic domain knowledge in parametrized action spaces, where each decision combines a discrete action with continuous parameters. We introduce our novel Knowledge- and Gradient-guided Reinforcement Learning (KGRL) algorithm that leverages on incomplete symbolic domain knowledge to reduce training time and improve safety during exploration. KGRL uses a Datalog knowledge base that encodes action restrictions and parameter bounds, which can be instantiated e.g. from domain expert knowledge. At each decision, the knowledge base is evaluated with a symbolic state abstraction, deriving state-specific action constraints and parameter bounds, while a gradient-based refinement searches the permitted parameter space for optimal parameter values. The symbolic derivations of the knowledge base additionally provide local traces that explain the imposed restrictions. Across six evaluation tasks in four different environments, KGRL achieves higher returns faster than five RL baselines for parametrized action spaces. Ablations show that including the incomplete domain knowledge accounts for substantial performance gains, whereas the benefit of the gradient guidance remains task-dependent.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Deep Reinforcement Learning, Neuro-Symbolic Reinforcement Learning, Datalog, Symbolic Reasoning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_W4tCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies reinforcement learning with incomplete symbolic knowledge in parametrized action spaces and proposes KGRL. A Datalog knowledge base derives state-dependent restrictions on discrete actions and bounds on their continuous parameters. PARAMOPT then refines the parameters using gradients of the learned Q-function. Knowledge constrains both action selection and TD targets, while symbolic derivations provide rule-based traces. The evaluation covers four environment families, five parametrized-action RL baselines, and ablations of knowledge guidance, gradient guidance, and their application at different stages.

Critical strengths
  1. A meaningful research problem. Complete symbolic environment models are often unavailable, while partial expert knowledge can still eliminate unreasonable decisions. Integrating such knowledge into RL with both discrete action choices and continuous parameters is a worthwhile direction.

  2. Broad experimental coverage. Multiple environments, five external baselines, eight random seeds, and multiple ablations help analyze the roles of knowledge and gradient guidance at different stages.

Critical weaknesses
  1. The incremental gains from GG are very small, leaving the added value of the combined design insufficiently established. In Table 1, removing GG during both training and evaluation produces results close to the complete method, with better mean performance on some tasks. The authors also acknowledge task-dependent benefits. The evidence therefore supports knowledge constraints more strongly than the necessity of gradient-based parameter refinement. This limits the demonstrated additional contribution of the combined design. The construction effort and cross-task reuse of the handcrafted rules and state abstractions are also not quantified.

  2. Baseline comparisons involve unequal access to knowledge and implementation correctness concerns.

    Unequal knowledge access: Most of KGRL's gains may come from the additional expert knowledge, while its algorithmic design may contribute relatively little. The external baselines do not receive equivalent knowledge, making the comparison unfair in this respect. I suggest adding some baselines that can also use external expert knowledge, such as P-DQN/MP-DQN augmented with expert knowledge, or other knowledge-guided RL/NeSy methods, including suitably adapted methods originally designed for non-parametrized action spaces. This would help establish whether KGRL provides additional gains.

    Implementation correctness: In the supplementary P-DQN and MP-DQN implementations, replay does not retain the executed continuous parameters. Critic updates instead recompute parameters using the current actor. The action supplied to the Q-function can therefore differ from the complete action that generated the sampled reward and transition. Relevant locations are lines 389–396 and 439–443 of the P-DQN training implementation, and lines 395–402 and 445–449 for MP-DQN. This raises concerns about baseline reliability, although I cannot determine whether the reported experiments used this version or quantify its impact.

Questions for authors
  1. Comparison with baselines using expert knowledge. Could the authors add representative baselines that can also use external expert knowledge, such as P-DQN/MP-DQN augmented with the same knowledge, or other knowledge-guided RL/NeSy methods, including methods originally designed for non-parametrized action spaces with suitable action-space adaptations? A representative subset would suffice; adapting every baseline is unnecessary. For knowledge-augmented P-DQN/MP-DQN, knowledge could likewise constrain action selection, continuous parameters, exploration, and TD targets under comparable budgets to test whether KGRL retains a consistent advantage. Other methods should also receive the same or equivalent knowledge.

  2. Clarification and correction of the baseline code in the supplementary material. The P-DQN and MP-DQN implementations in the supplementary code do not store the executed continuous parameters in replay and instead recompute them using the current actor during critic updates. Is this intentional? Were the experiments reported in the paper conducted using these implementations from the supplementary material? Please explain the justification or, if erroneous, provide corrected comparisons.

  3. Evaluation in more complex environments. The current evaluation environments are relatively simple. Could the authors add an experiment on a more challenging environment with parametrized actions, for example one with richer visual observations, longer episodes, or larger state and action spaces, to assess whether KGRL's performance advantages persist?

2 comments

We thank the reviewer for the careful review. Most importantly, we corrected the implementations and added comparisons under identical knowledge, alternative estimators. Therefore, we added substantial experiments with three new evaluation domains that describe navigation task with obstacles (Bottleneck, ZigZag, and UNav) to the Appendix of the updated paper. These experiments are only performed on 3 seeds and 300k trainin steps. A full sweep across 8 seeds and 500k training steps is currently running and we will update the results in the main paper and appendix accordingly, as soon as it is finished.

Weakness 1. Contribution of gradient-guidance and handcrafted knowledge “The incremental gains from GG are very small.” “Most of KGRL's gains may come from the additional expert knowledge.”

We added P-DQN-KB and MP-DQN-KB with identical knowledge (Appendix B.2, Table 5, Figure 5). On UNav, KGRL first reaches 100% success at 20.0±0.0k interactions, versus P-DQN-KB's 73.3±49.3k and MP-DQN-KB's 90.0±10.0k. Its mean AULC is 9.663±0.119 versus 8.845±0.642 and 8.167±0.389, respectively. On ZigZag, KGRL and MP-DQN-KB reach 100% final success with comparable AULC, while P-DQN-KB reaches 65.6±42.0%. MP-DQN-KB is stronger on Bottleneck, which shows that the advantages are domain-dependent.

Weakness 2. Baseline implementation correctness “Replay does not retain the executed continuous parameters.”

We confirmed and corrected this error in our local adaptations of P-DQN and MP-DQN. The replay buffer now retains the parameter vector, and critic regression uses the action that generated each reward and transition. We also corrected MP-DQN's action-specific multi-pass outputs. Appendix B.1, Table 3 reports fresh comparisons using the corrected implementations under a shared 300k-interaction budget. KGRL achieves higher mean return AULC than both corrected baselines on eight of nine tasks. Platform is the exception, where corrected MP-DQN is stronger.

Question 1. Comparisons with knowledge-augmented baselines?

P-DQN-KB and MP-DQN-KB apply the same knowledge guidance during exploration, deployment. Table 5 and Figure 5 in the appendix report the results. Our answer to Weakness 1 reports KGRL's remaining advantage.

Question 2. Was recomputing replay parameters intentional, and which code produced the original results?

It was an unintentional error in our local P-DQN and MP-DQN adaptations. We corrected the error as described in our answer to Weakness 2. The original evaluation logs do not preserve the training-source version, so those records cannot establish which implementation produced the original results. The new results in Table 3 use the corrected code and independently support the comparison.

Question 3. Evaluation in more complex environments?

We introduced three additional diagnostic domains (Bottleneck, ZigZag, and UNav), described in Appendix A.1. The required trajectories to solve the domains are substantially longer than the ones of the benchmark domains. ZigZag requires successive detours around alternating barriers, UNav requires temporarily moving away from the goal to enter the obstacle's opening. At 300k interactions, KGRL achieves 95.8±4.8%, 100±0%, and 100±0% final success. The vanilla version of MP-DQN reaches 0% on all three domains, and P-DQN reaches 1.0±1.8%, 0%, and 0% (Table 3). Figure 8 additionally shows 0/288 final collision episodes for KGRL versus 213/288 for MP-DQN and 112/288 for P-DQN. The equal-knowledge results in our answer to Weakness 2 separate the contribution of supplied guidance from KGRL's additional learning benefit.

Thank you for correcting the baseline code, comparing methods with the same expert knowledge, and testing on harder tasks. GG’s added benefit remains unclear, but I’ve decided to raise my score.

Reviewer_dyM1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a novel neurosymbolic algorithm (KGRL) to solve a Parametrized Action Markov Decision Process (PAMDP) that adds an incomplete knowledge base whose rules are assumed to be correct, and a gradient-guided parameter refinement for the action parameters. Experimental results show that the area under the learning curve of KGRL is larger than those of other five baselines. Ablation studies show that the knowledge base is more favorable during deployment than during training. The datalog also seems to provide information to support explainability.

Critical strengths

KGRL has a conceptually simple and intuitive approach to integrating symbolic domain knowledge with reinforcement learning. The proposed framework is relatively easy to understand, and KGRL appears straightforward to implement. The paper delivery is clear and easy to follow.

Critical weaknesses

The learning curves do not clearly show faster learning or better sample efficiency as stated in the paper. It can be due to escale, but the improvement shown is not convincing in the learning curves. Additionally, improved performance is clearly shown in only two domains, in HardMove4 and HardGoal, KGRL shows similar performance to MP-DQN. The metric used (AULC) assesses an aggregated performance of both the sample efficiency and improved performance, but the authors need to better assess the outcomes to claim that KGRL outperforms all baselines and that it improves sample efficiency.

The ablation studies show the importance of the KB. However, it is not clear whether or how the amount and quality of knowledge given affects the performance. Those variables (amount and quality of knowledge) should become critical experimental variables because the paper specifically focuses on incomplete knowledge.

Questions for authors
  1. Since AULC combines learning speed and achieved performance, what evidence supports the conclusion that KGRL improves sample efficiency, and it is not mainly achieving higher returns? Could the authors provide a more direct analysis of sample efficiency?

  2. What is the intended use of the explanation traces, and how would their usefulness be evaluated?

  3. How incomplete are the knowledge bases used in the evaluated domains, and how does the amount and informativeness of the provided knowledge affect KGRL's performance?

1 comment

We thank the reviewer for the careful review. We added direct sample-efficiency measurements, experiments on individual knowledge rules, and worked explanation examples. We added experiments with three new diagnostic domains that describe navigation task with obstacles (Bottleneck, ZigZag, and UNav) to the Appendix of the updated paper. These experiments are only performed on 3 seeds and 300k trainin steps. A full sweep across 8 seeds and 500k training steps is currently running and we will update the results in the main paper and appendix accordingly, as soon as it is finished. The additional results show how knowledge guidance improves learning and which rules contribute to this improvement.

Weakness 1. Sample efficiency and AULC "AULC assesses an aggregated performance of both the sample efficiency and improved performance."

We disentangled AULC to success rate and final return. Additionally, we evluated success and return at fixed interaction budgets during training (Appendix B.1). After 10k HardMove4 interactions, KGRL reaches 82.3±28.0% success, versus 10.4±4.8% for MP-DQN and 6.2±5.4% for P-DQN. At 300k, all three reach 100% success (Table 3). This directly shows an advantage early in learning at the same interaction budget, even though the methods eventually reach the same task-completion rate.

The disentangled comparison also extends across the other domains. KGRL has higher mean return AULC than both P-DQN and MP-DQN in eight of them (Table 3). On HardGoal, its final return is 49.562±0.758 versus MP-DQN's 44.521±3.459 and P-DQN's 48.854±0.321. Platform is the exception, where MP-DQN performs better.

Weakness 2. Amount and informativeness of knowledge “It is not clear whether or how the amount and quality of knowledge given affects the performance.”

We added experiments with knowledge ablation on the new diagnostic domains (Appendix D.1, Table 7, Figure 7):

ComparisonReturn AULCFinal success
ZigZag, full guidance → omit middle-barrier rule9.411 → 2.84996.9% → 40.6%
UNav, full guidance → omit obstacle-passing rule9.661 → 3.536100% → 81.2%

The comparisons show that specific navigation knowledge has a substantial effect on learning and successful execution. Other rules mainly affect learning progress. E.g. removing ZigZag's goal-directed action pruning preserves 100% final success, but lowers AULC from 9.411 to 6.354. Only keeping safety specific rules lets all runs finish at 0% success. This shows that safe movement alone does not supply sufficient navigation guidance in these runs.

Question 1. Direct evidence of sample efficiency beyond higher final returns?

The fixed-budget result in our answer to Weakness 1 provides evidence to this question. We also controlled knowledge access by equipping P-DQN and MP-DQN with KGRL's KB (Appendix B.2, Table 5, Figure 5). On the UNav domain, KGRL first reaches 100% success at 20.0±0.0k interactions, versus 73.3±49.3k for P-DQN-KB and 90.0±10.0k for MP-DQN-KB. All algorithms finish at 100%. However, KGRL reaches the same success level earlier, even including equally informed baselines.

Question 2. Intended use of explanation traces?

We added a worked out example to the Appendix (Appendix E). The example connects state facts, activated rules, and decision restrictions (Appendices E.1–E.3, Figure 9) in a UNav deployment. In the example, the explanation trace explains why the agent must pass the obstacle before approaching the goal and why a nearby wall restricts downward movement. A counterfactual position farther from the wall restores a distance bound while leaving the action mask unchanged. Alignment with the opening instead enables upward movement. These queries use the same KB throughout, making the conditions for changing a restriction explicit and checkable. The trace explains the symbolic part of the decision process. Section 5.2 measures its frequency of use. Broader aggregation and human evaluation remain future work, as discussed in Section 6.

Question 3. How incomplete are the knowledge bases?

Appendix A.1 now lists each domain's abstraction features, pruning rules, and parameter constraints. They provide partial restrictions and navigation guidance without specifying complete transition or reward models. Default applicability leaves unspecified actions available. Experiments on Weakness 2 then test the informativeness of the supplied rules directly. By removing particular route rules substantially AULC and success are reduced significantly, while some other omissions preserve final success. Different ablations on omitting rules in the knowledge base are evaluated in Appendix D.1 and Figure 7.

Reviewer_dbegOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes KGRL for parameterized action spaces. A Datalog knowledge base restricts discrete actions and continuous parameter regions, while projected gradient ascent refines parameters using a learned Q-function. Rule derivations provide local procedural explanations. The formulation is understandable and practically relevant. However, the main experiments support the usefulness of expert constraints more convincingly than the necessity of the complete algorithm.

Critical strengths
  1. Default applicability provides a natural mechanism for incorporating incomplete knowledge without requiring a complete domain model.

  2. Knowledge constraints are integrated into exploration, deployment, and successor-target computation.

  3. The experiments separate training-time and evaluation-time removal of knowledge and gradient refinement, producing useful evidence about when each component matters.

  4. The discussion appropriately limits explanation traces to rule-grounded restrictions rather than claiming to explain the Q-network’s full decision process.

  5. Additional appendix experiments on larger action sets and incomplete or incorrect rules provide useful supporting evidence.

Critical weaknesses
  1. Gradient-guided refinement does not show a consistent benefit. In Table 1, removing training-time refinement improves HardMove4 AULC from −1.64 to −1.43. Removing refinement during both training and evaluation slightly improves CatchPoint and leaves Platform essentially unchanged. The necessity of this computationally expensive component remains unclear.

  2. The principal baselines do not receive the same knowledge. Comparisons against unconstrained parameterized-action algorithms cannot fully separate the benefit of expert restrictions from the benefit of the proposed integration mechanism. A strong existing algorithm equipped with identical action masks and parameter bounds would be a more direct test.

  3. Sample-efficiency improvements are not sufficiently connected to computational cost. PARAMOPT operates during action selection and learning-target construction, and the discussion acknowledges its additional computation. The main results emphasize environment-step performance, leaving uncertainty about the practical return on this extra computation.

  4. Safety and exceptional-state handling need clearer evidence. Knowledge feasibility is not equivalent to environmental safety when the knowledge is incomplete. Moreover, the implementation resets without storing a transition when all actions are excluded. The frequency and accounting of these resets could materially affect learning and reported performance.

Questions for authors
  1. How much improvement remains over MP-DQN or P-DQN supplied with the same knowledge-derived constraints?

  2. Does PARAMOPT outperform equal-compute random search, multiple-start optimization, or a learned parameter estimator?

  3. What are the wall-clock learning curves and action-selection costs?

  4. How often are all actions excluded, and how are these resets reflected in interaction budgets and returns?

  5. Can safety-related claims be supported by direct violation or hazard measurements?

2 comments

We thank the reviewer for the careful review. We added experiments that separate the contribution of knowledge, parameter refinement, and computation to the appendix. Therefore, we added three new diagnostic domains that describe navigation task with obstacles (Bottleneck, ZigZag, and UNav). These experiments are only performed on 3 seeds and 300k training steps.

Weakness 1: Contribution of gradient-guidance “Gradient-guided refinement does not show a consistent benefit.”

Additionally to ParamOpt, we evaluated multi-start gradient-ascent, random search, and a learned one-shot estimator on the three diagnostic domains (Appendices C.1–C.2, Table 6, Figure 6). The highest mean return AULC in each domain is obtained by a gradient-based variant, underlining the advantage of using gradient-based approaches for parameter estimation.

DomainGradient refinementRandom searchOne-shot
Bottleneck7.733±0.087 (multi-start)7.276±0.6516.589±0.215
ZigZag9.554±0.159 (multi-start)9.068±0.2628.335±0.403
UNav9.663±0.119 (single-start)8.698±0.1918.479±0.609

These results show the contribution of gradient refinement when optimizing two continuous movement parameters. The best initialization strategy depends on the domain. On UNav all four estimators eventually reach 100% success, so the AULC advantage reflects their performance during learning.

Weakness 2: Unequal access to knowledge “The principal baselines do not receive the same knowledge.”

We added knowledge-guided P-DQN and MP-DQN with the same KB as KGRL (Appendix B.2, Table 5, Figure 5). On UNav, KGRL first reaches 100% success at 20.0±0.0k interactions, versus P-DQN-KB's 73.3±49.3k and MP-DQN-KB's 90.0±10.0k. Mean AULC is 9.663±0.119 versus 8.845±0.642 and 8.167±0.389, respectively. On ZigZag, KGRL and MP-DQN-KB reach 100% final success with comparable AULC, while P-DQN-KB reaches 65.6±42.0%. MP-DQN-KB performs better on Bottleneck.

Weakness 3. Computational cost “Sample-efficiency improvements are not sufficiently connected to computational cost.”

We report training time, total time, and action-selection latency on an NVIDIA A100 (Appendix B.1, Table 4). On the UNav domain, KGRL uses 2.74±0.02 training hours and 39.00±1.76 ms per final-evaluation decision. Though this might seem much, it achieves 100% success, whereas unconstrained P-DQN and MP-DQN remain at 0% (Table 3). This points to a performance--computation tradeoff: Though KGRL uses substantial computation, it supports learning agents that can operate in such domains.

Weakness 4. Safety and exceptional states “Knowledge feasibility is not equivalent to environmental safety when the knowledge is incomplete.”

We acknowledge that an absolute safety claim with incomplete knowledge is not possible. Formulating safety as conditional to the incomplete knowledge base would be more precise. For our experiments, we additionally measured collisions of the different approaches in the diagnostic domains (Appendix D.2, Figure 8). At the final evaluation checkpoints, KGRL and both KB adapters recorded 0/288 collision episodes, compared with MP-DQN's 213/288 (74.0%) and P-DQN's 112/288 (38.9%) collisions. KGRL also showed no observed collisions during training. The shared knowledge layer therefore improves measured safety across algorithms, however always conditional to the knowledge implemented in the knowledge base.

Question 1. Improvement under identical knowledge? Weakness 2 reports earlier success and higher mean AULC on UNav, comparable performance on ZigZag, and MP-DQN-KB's stronger Bottleneck results. The complete comparison can be found in Table 5.

Question 2. Alternative estimators and equal compute? Table 6 compares all requested estimator types. Gradient-based variants obtain the highest mean AULC in each domain (cf. Weakness 1). Budgets are matched by environment interactions, not calibrated to equal computation.

Question 3. Wall-clock learning and selection costs? Table 4 gives measured training/total hours and decision latencies. Figures 5–6 plot learning against interactions.

Question 4. How frequent are exclusions, and how are resets counted? The saved run records contain 1,930 KGRL training conflicts on HardMove4 across three seeds, out of 901,930 policy requests (0.214%). On HardMove4/8/10 there are 5,896 KGRL conflict resets and 3,725 KB-sampler resets, totaling 9,621. No evaluation policy request is fully excluded. Every run still completes 300k actual interactions. Resets create no replay transition and consume no interaction. Their accounting is defined in Section 4.1 and Appendix A.2.

Qestion 5. Direct safety evidence? Figure 8 reports collision episodes with explicit denominators, including the trained policies' final evaluation (cf. Weakness 4).

Thanks for the additional experiments and clarification. It's good to see the baseline implementation issues corrected and the experiments rerun. The new knowledge-matched comparisons are particularly useful.

I'm still concerned that KGRL's advantage over methods using the same knowledge is inconsistent across tasks, and the benefit of gradient refinement remains unclear relative to its computational cost. I'm currently on the borderline between reject and accept.

I'll keep my rating at Weak Reject for now but I'm ok if other reviewers agree with acceptance and the paper is accepted.

Comments on the paper

1 comment

We thank all reviewers for their careful reviews. Following their comments, we extended the evaluation of our proposed algorithm. The updated appendix contains the new and revised experiments listed below. The results are reported to the updated paper's appendix and in the individual responses.

Changes we applied:

  • Corrected baselines (Reviwer 3). We corrected an error in our implementations of P-DQN and MP-DQN that was pointed out by Reviewer 3. The replay buffor now retains the continuous parameters, and MP-DQN's multi-pass outputs are action-specific. All baseline results in Table 3 were rerun with the corrected code. KGRL obtains the highest mean return AULC on eight of nine tasks; on Platform, MP-DQN performs better.

  • Navigation domains (Reviwer 3). We introduced three new diagnostic domains. The domains describe navigation tasks with obstacles: Bottleneck, ZigZag, and UNav. The Bottleneck domain requires an agent to pass a bottleneck in a barrier. The ZigZag domain requires an agent to navigate around three barriers. The UNav domain requires an agent to navigate around a U-shaped obstacle forcing it in a not-goal-directed movement before being able to reach the goal (Appendix A.1, Figure 4).

  • Knowledge-equal baselines (Reviewer 1, Reviewer 2, Reviewer 3). We evaluated knowledge-guided versions of P-DQN and MP-DQN that reveive the same knowledge base as KGRL. This allowed us to estimate the influence of the employed knowledge guidance. KGRL obtains the highest mean AULC on UNav. On ZigZag, KGRL and MP-DQN-KB reach 100% final success with comparable AULC. On the Bottleneck domain, the knowledge guided version of MP-DQN performs better.

  • Alternative parameter estimators (Reviewer 1, Reviewer 3). We additionally evalutated multi-start gradient ascent, random search, and a learned one-shot estimator compared to our ParamOpt algorthm (Appendices C.1–C.2, Table 6, Figure 6).

  • Fixed-budget success rates (Reviewer 2). We additionally evaluated the success rate after fixed numbers of interactions, separating learning speed from final performance (Appendix B.1).

  • Computational cost (Reviewer 1, Reviewer 3). We additionally evaluated time, total time, and action-selection latency on a NVIDIA A100 (Appendix B.1, Table 4).

  • Rule ablations (Reviewer 2, Reviewer 3). We additionally evaluated rule ablatoins of individual knowledge rules in the diagnostic domains (Appendix D.1, Table 7, Figure 7).

  • Collision measurement (Reviewer 1). We additionally evaluated collisions episodes during training and in the final evaluation (Appendix D.2, Figure 8).

  • Usage of explanation traces (Reviewer 2, Reviewer 3). We added a worked explanation example with counterfactual queries to our appendix (Appendices E.1–E.3, Figure 9).

All new results are based on 3 seeds and 300k environment interactions per run. A sweep with 8 seeds and 500k interactions is currently running. We will update the main paper and appendix as soon as it has finished.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.