acceptodds
Under review as a conference paper at ICLR 2027

CAPER: Closing the Loop between LLM Action Priors and Reinforcement Learning

Abstract

Reinforcement learning (RL) often requires extensive environment interaction, while large language models (LLMs) can provide action priors that guide exploration using pretrained knowledge. However, fixed LLM priors may be mismatched with the environment and cannot adapt when grounded failures reveal unsuitable guidance. We propose losed-loop ction rior nhancement for einforcement Learning (CAPER), which alternates between RL policy learning and failure-guided action-prior refinement. Low-return trajectories identify weaknesses in the current policy–prior pair, which guide the retrieval of relevant task knowledge and the generation of revised prompt candidates. Candidate prompts are selected either through direct return evaluation or through agreement with actions in high-return trajectories, trading evaluation fidelity for environment-interaction cost. Across five tasks from three environments and multiple RL settings, CAPER consistently improves over baselines under total environment-interaction budgets. The implementation is available at https://anonymous.4open.science/r/CAPER-FD27.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_waPKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework named CAPER, which achieves improvement by alternately using reinforcement learning (RL) policy learning and failure-based LLM action prior optimization. This method identifies the weak points in the current policy-prior pair through low-return trajectories, retrieves relevant task knowledge, and generates revised prompt candidates for subsequent RL training. Two prompt selection strategies are considered in the paper: the reward-based selection (RBS) evaluates candidate solutions through environmental interaction, while the trajectory consistency selection (TAS) selects based on the consistency of prompts with actions in high-return trajectories. During training, the LLM remains in a frozen state.

Critical strengths
  1. A reasonable framework for adaptively incorporating LLM action priors during reinforcement learning training.
  2. Extensive experimental evaluation, including useful component analysis.
Critical weaknesses
  1. The connection between policy failure and adjustment of action priorities has not been fully established.
  2. The contribution of closed-loop optimization to additional task knowledge and computational costs needs to be more clearly defined.
Questions for authors

Q1. Failure Attribution and Action Prior Quality.

Can the author analyze whether the low-return trajectories are mainly caused by the missing candidate actions in the LLM prior, or by the incorrect selection of the reinforcement learning strategy? In particular, measuring the coverage of the optimal action in the candidate action set and the accuracy of the strategy selection separately can help clarify the contribution of prior optimization.

Q2. RBS Refinement Trigger.

In Section 3.3.1, when the current return exceeds the previously stored reference return, RBS will trigger a new round of optimization. Why this condition was chosen instead of triggering optimization when performance stagnates or declines? Since the strategies change between each round of optimization, how does this method distinguish between strategy improvements and improvements in action priors?

Q3. Task Knowledge and Computational Efficiency.

The optimized prompt statement incorporates detailed task-related knowledge extracted from the environment description and the source code. Can the author further quantify the extent to which the performance improvement brought about by the guided optimization through the optimization process is derived from the optimization process itself, compared to providing highly task-oriented prompts from the very beginning? Furthermore, Table 10 shows that the training time cost of CAPER is significantly higher than that of DQN-Prior. Can the author provide a more detailed breakdown of the costs for LLM reasoning, retrieval, candidate selection, and reinforcement learning optimization, especially for TAS, which although avoids additional environment runs, still requires repeated generation of LLM?

Q4. Fixed-Dataset Policy Learning.

The CAPER-CQL experiments use a fixed dataset during policy updates, but collect additional online trajectories for action prior optimization. Can the author further explain the envisioned offline reinforcement learning setup and evaluate whether TAS can achieve prior optimization solely through the fixed dataset, without the need for additional interaction with the environment? Such comparisons can help verify the applicability of CAPER when access to the online environment is unavailable.

Q5. Interpretation of the Finite-Sample Guarantees.

Proposition 3.1 provides a selection guarantee under the corresponding overall criterion. However, for TAS, this criterion measures consistency with high-reward trajectories, rather than the expected environmental rewards. Can the author explain under what conditions maximizing trajectory consistency can lead to an improvement in downstream benefits? This distinction is of great significance for understanding the theoretical contribution of TAS.

4 comments

We sincerely thank the reviewer for the constructive feedback on our adaptive prior framework. We respond below with the additional evidence and clarifications.

Q1: Failure attribution

A1: Thank you for suggesting this analysis. We compare prompts before and after the first accepted refinement in three CAPER-DQN (TAS) runs on TWC, using 136 state-history inputs from low-return trajectories in total. Each run holds the inputs and Q network fixed. Correct actions follow the task's object-placement rules. The table reports candidate coverage, accuracy conditional on a correct candidate being available, and overall accuracy under the evaluation policy, as percentages with means and standard deviations across runs. Missing candidates fall from 28.93% to 15.17% of decisions, while incorrect selection despite an available correct candidate changes from 16.68% to 16.50%. Improved coverage thus accounts for most of the 13.94-percentage-point gain in overall accuracy. These results are in Appendix B.15.3, Table 12, and Section 4.3.

MetricBefore refinementAfter refinement
Candidate coverage71.07 ± 3.6484.83 ± 5.06
Conditional selection accuracy76.34 ± 10.5980.53 ± 6.77
Overall selection accuracy54.40 ± 9.5168.33 ± 7.49

Q2: Refinement trigger

A2: Thank you for suggesting refinement when progress stalls. We tested a non-improvement trigger on Salad: after initialization, refinement starts when the current mean return does not exceed the preceding measurement. Under the same RL-training budget as the original method, this variant achieves 0.959 ± 0.239, reported as mean ± standard deviation across runs. Compared with the return of 0.98 for original CAPER-DQN (RBS) (Appendix B.4, Figure 10), this result indicates that both approaches support effective learning, although their motivations differ. Original RBS refines unconditionally at initialization, then triggers when the current policy–prior return exceeds the return stored when the preceding prompt was selected. It uses failures remaining after policy progress to guide further refinement. The alternative seeks new guidance when progress stalls. The joint return determines when to refine, while comparisons with the policy held fixed assess the contribution of changes to the prior, as illustrated by the coverage and selection-accuracy analysis in Q1.

Q3: Continued refinement beyond initialization

A3: Thank you for raising the role of initialization. Static Full-KB Prior + DQN distills all task knowledge into a fixed guide before training, yet CAPER achieves higher returns (Figure 8(b)). Single Refinement freezes the prompt after its first refinement. Continuing refinement on TWC raises return from 4.08 to 4.67 for RBS at 290 total environment episodes, and from 2.64 to 5.35 for TAS at 260 episodes, with equal budgets within each selector (Appendix B.7, Figure 15). CAPER also outperforms w/o Failure Feedback, which retains the knowledge base and refinement components but replaces failures with the task description (Figure 8(a)). These controls support continued, experience-guided revision beyond knowledge-based initialization.

Q4: Computational costs

A4: Thank you for asking about these costs. Appendix B.15.6, Table 16 provides a detailed time breakdown for CAPER-GFlan (TAS) in Table 10. The logged total is 41.16 minutes, corresponding to 0.69 hours in Table 10. TAS thus replaces candidate-specific environment validation with repeated LLM computation on stored trajectories. Separately, the fixed-input benchmark on one RTX 4090D reports average five-action generation latencies of 0.948 seconds for RBS and 0.947 seconds for TAS on inputs from TWC (Appendix B.15.6, Table 15).

Q5: Fully offline refinement

A5: Thank you for suggesting this variant. We tested fully offline TAS on Salad, drawing both low-return refinement feedback and high-return selection references from 51 fixed trajectories with 1,030 transitions. This variant follows the original TAS training configuration, but uses only trajectories from the fixed dataset for prompt refinement and candidate scoring. Policy training and prompt adaptation use zero additional environment episodes. As shown below, fully offline TAS achieves a higher mean return than CQL-Prior using fixed experience alone. We also include original TAS with online prompt adaptation for comparison. Values are means and standard deviations. The comparison is in Appendix B.15.5, Table 14, and Section 4.2.

MethodReturn on Salad
CQL-Prior0.34 ± 0.10
CAPER-CQL (TAS), online prompt adaptation1.09 ± 0.18
CAPER-CQL (TAS), fully offline0.95 ± 0.17

Q6: Interpretation of the selection guarantee

A6: Thank you for distinguishing agreement from return. For TAS, Proposition 3.1 places the selected prompt's population agreement score within 2ε of the best candidate when estimation errors are uniformly bounded by ε. Agreement can guide learning when reference trajectories cover relevant states, their actions remain useful, and the policy can exploit those actions. We tested the score–return relationship with one saved DQN policy fixed per task. We sample eight prompts with different returns using equally spaced return targets, then compute the TAS score for each prompt. Spearman correlation measures agreement between score and return ranks, computed as their Pearson correlation with average ranks for ties. Ten TAS scoring repetitions against the same returns yield positive correlations throughout. The table gives their mean and standard deviation (Appendix B.15.4, Table 13; Section 4.5). Figure 17 also shows higher returns with TAS than random prompt selection under identical components and interaction budgets. Together with Q1's coverage analysis, these results support trajectory agreement as a refinement criterion.

TaskSpearman correlation
Overcooked Salad0.67 ± 0.09
VirtualHome Food Preparation0.62 ± 0.14
TextWorld-Commonsense0.71 ± 0.13

We appreciate the reviewer's comments, which helped us improve the clarity and completeness of the paper.

The rebuttal addresses most of my concerns. Some limitations remain regarding the RBS trigger (Q2) and the scope of the theoretical guarantee (Q5). Overall, I am satisfied with the responses and maintain my original rating.

Thank you for your follow-up. We would like to clarify the two remaining points.

Regarding the RBS trigger (Q2), between refinement rounds, the prompt stays fixed while the policy learns. The trigger therefore detects progress under unchanged guidance. Once triggered, the current prompt and new candidates are compared with the same policy held fixed, separating prompt changes from concurrent policy updates. We choose progress-triggered refinement to address remaining failures as learning advances, rather than wait for stagnation. Non-improvement-triggered refinement is also a viable alternative, as supported by our Salad experiment reported in our initial response.

Regarding the scope of the theoretical guarantee (Q5), TAS favors prompts that propose actions found in high-return reference trajectories. This can help when those trajectories cover states relevant to the current policy, their actions remain useful, and the policy can select them from the candidate set. Proposition 3.1 bounds selection error under this agreement criterion. The practical relevance of this agreement criterion is supported by the positive Spearman correlations in Table 13, the fixed-policy coverage analysis in Table 12, and the TAS-versus-random comparison in Figure 17.

Thank you again for your feedback. We hope these clarifications address your remaining concerns about the RBS trigger and the interpretation of the TAS guarantee.

Reviewer_P41zOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes CAPER, a framework that jointly improves an RL policy and its LLM-generated action prior. Unlike prior methods that use a fixed LLM to propose candidate actions, CAPER periodically uses failed RL trajectories to identify weaknesses in the prior, retrieve relevant task knowledge, and revise the prompt. Two selection strategies are proposed: Return-Based Selection (RBS), which evaluates candidate prompts through environment rollouts, and Trajectory-Agreement Selection (TAS), which selects prompts based on agreement with high-return trajectories without additional evaluation rollouts. Experiments cover five tasks across Overcooked, TextWorld-Commonsense, and VirtualHome, using DQN, PPO, and CQL backbones. CAPER consistently improves over fixed-prior baselines under matched environment-interaction budgets. Ablations examine the contributions of failure feedback, retrieval, repeated refinement, and prompt selection.

Critical strengths
  1. Addressing the action-prior bottleneck. Improving the RL policy alone cannot help if useful actions are missing from the candidate set. Updating the action prior based on RL experience is a sensible approach to this problem. The improvement in optimal-action sampling rates (Fig. 6) also provides evidence that CAPER improves the prior itself, rather than just the RL policy.

  2. Solid empirical validation. The improvements are consistent across different RL backbones and tasks. I also appreciate the ablations on failure feedback, RAG, and repeated refinement, as well as the comparisons with adaptive prompt baselines. These make the experimental results more convincing.

Critical weaknesses
  1. Reliance on privileged environment knowledge. The knowledge base is constructed partly from environment source code and includes action preconditions, task procedures, and example trajectories. This provides unusually strong task-specific guidance. While the ablations support the benefit of failure-guided retrieval, it is less clear whether CAPER remains effective when such detailed knowledge is unavailable. This limits the generality of the approach.

  2. Limited justification for TAS. Agreement with high-return trajectories does not necessarily imply that a prompt will produce better downstream RL performance. The theoretical analysis bounds estimation error for the agreement score, but does not establish its relationship to return. The random-selection ablation is useful, but a more direct validation of this surrogate would strengthen the contribution.

Questions for authors
  1. How does CAPER perform when the knowledge base contains only general task descriptions, without environment source code or example solution trajectories?

  2. How strongly does the TAS score correlate with actual environment returns across candidate prompts? Does this relationship hold consistently across tasks?

2 comments

We sincerely thank the reviewer for the constructive comments and for recognizing the motivation and empirical validation of our adaptive prior framework. We respond below with additional evidence on the knowledge requirements and the relationship between TAS scores and environment returns.

Q1: Knowledge requirements

A1: Thank you for suggesting this experiment. We tested CAPER on Overcooked Salad after removing source code and successful solution examples from the knowledge base. CAPER continues to outperform DQN-Prior under both RL-training and total environment-interaction budgets. It remains competitive with the original-knowledge variant, showing that prior refinement remains effective without source code or successful solution examples in this setting. We have added the comparison in Appendix B.15.2, Figure 20, and summarized it in Section 4.5.

Q2: TAS scores and environment returns

A2: Thank you for suggesting this analysis. On each of the three tasks below, we fix one saved DQN policy and sample eight prompts with different returns using equally spaced return targets, then compute the TAS score for each prompt. Spearman correlation is the Pearson correlation between score and return ranks, assigning average ranks to ties; positive values indicate that higher TAS scores tend to accompany higher returns. All ten TAS scoring repetitions against the same returns yield positive correlations in each task; the table reports their means and standard deviations. TAS thus captures return-related ranking information across these prompt sets. We have added the results and protocol in Appendix B.15.4, Table 13, with a summary in Section 4.5.

TaskSpearman correlation
Overcooked Salad0.67 ± 0.09
VirtualHome Food Preparation0.62 ± 0.14
TextWorld-Commonsense0.71 ± 0.13

We sincerely appreciate the reviewer's time and thoughtful suggestions, which have helped us improve the clarity of the paper and strengthen its experimental evaluation.

Thank you for the new results. It addresses my concerns. I will keep my positive rating.

Reviewer_gAQAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an adaptive prompting mechanism for LLMs in RL. The authors build their method based on DQN-Prior, where LLM helps prune the action space, and intuitively RL learns faster with the smaller action space. In DQN-Prior, the prompt for action pruning is fixed while in this paper, the authors propose to adaptively improve the prompt. They collect the environment source code and the three trials with worst RL returns as the context. An LLM improves the prompt based on the collected context.

Critical strengths
  1. The authors present clear accounting of the rollout cost in RL environments for adapting prompts.
  2. The authors provide open-source codebase and prompts. In general, the engineering efforts are sufficient and the experiments are sound.
Critical weaknesses

This is a paper with fatal flaws.

  1. Unclear, weak motivation and problem framing. The paper argues that the fixed prompt may be mismatched with the environment and cannot adapt when grounded failures reveal unsuitable guidance. However, there is a straightforward solution to the research problem here: simply appending environment information (text description, source code, etc) and the failure history into the prompt context. For example, in Appendix B.14.2, the initial prompt (deliberately designed to be suboptimal) only describes the target, while the prompt by CAPER has the env description (e.g. "7x7 Overcooked grid-world environment") and various task instructions (e.g. "tomato and lettuce"). These additional prompting sentences are accessible in the env source code / paper. With all the wrapping, algorithms, the paper proposes something like "just add the user manual of the env to your prompt".
  2. I have significant concerns about the reported baseline performance and the article contains overclaims. In table 1, on TextWorld-Commonsense, BC and CQL are both 0.00 ± 0.00. On the overcooked (salad), BC achieves −0.03 ± 0.00, which is (31 steps \times -0.001 step penalty). The policy never completes a single useful action. BC on expert-only data should be a strong or at least mediocre baseline. These suggest an implementation problem, not a methodological gap.
Questions for authors
  1. What is "action projection" (line 140)?
4 comments

We sincerely thank the reviewer for the detailed and constructive comments, and for recognizing our efforts in interaction accounting and reproducibility. We respond below with additional experiments and clarifications addressing the main concerns.

Q1: Direct knowledge and failure history

A1: Thank you for suggesting this comparison. Direct-KB Prior + DQN supplies the complete indexed knowledge at each decision; Direct-KB + History Prior + DQN also supplies three recent low-return trajectories and a reminder to avoid repeating failures. Both retain DQN; like CAPER, the second uses failure feedback, but supplies it directly. Adding history does not improve this baseline. Under the same RL-training budget on TWC, CAPER-RBS reaches performance close to Direct-KB at the common endpoint, while CAPER-TAS achieves a higher endpoint mean return (Appendix B.15.1, Figure 19). Under the same total environment-interaction budget, TAS outperforms both direct-input baselines, while RBS remains below Direct-KB at the common endpoint (Section 4.5, Figure 7), but requires substantially less time per action-generation call. The table below reports mean input lengths and per-call generation latencies using the same model, decoding settings, and fixed state-history inputs from TWC on one RTX 4090D (Appendix B.15.6, Table 15). CAPER uses about 73% and 84% fewer input tokens than Direct-KB and Direct-KB + History, respectively.

MethodInput tokensGeneration latency (s)
Direct-KB Prior + DQN3,1902.96
Direct-KB + History Prior + DQN5,5285.01
CAPER-DQN (RBS)8760.95
CAPER-DQN (TAS)8720.95

Q2: Strong initialization and continued refinement

A2: Thank you for raising this concern. Our comparisons test whether stronger initialization can replace continued refinement. Static Full-KB Prior + DQN summarizes the complete knowledge base into a fixed prompt before training, yet CAPER achieves higher returns (Figure 8(b)). Single Refinement instead keeps the first refined prompt fixed; continued refinement also outperforms this control under the same total interaction budget for each selector (Appendix B.7, Figure 15). These results support adapting the prior to failures encountered as the policy learns, rather than relying on a stronger initial prompt alone. The Salad case study provides complementary evidence: even when the initial prompt already specifies the ingredients and procedure, revising it with DQN held fixed increases mean return from 0.57 to 0.75 (Appendix B.11, Table 9).

Q3: Baseline implementation and hyperparameters

A3: Thank you for pointing this out. CQL initially inherited hyperparameters tuned for CAPER-CQL. A pilot comparison on Salad gave returns of 0.03, 0.45, and 0.44 for conservative coefficients of 0.5, 8, and 64, respectively. We adopted 8 as the best-performing tested setting and reran CQL-Prior and both CAPER-CQL variants with the same coefficient. We also corrected a BC-specific action-label indexing error and reran BC; the error did not affect other methods or any original CAPER results. Under the shared coefficient, CQL-Prior achieves 0.39 ± 0.09, 1.47 ± 0.40, and 0.82 ± 0.72 on Overcooked Salad, TWC, and VirtualHome Food Preparation, respectively; CAPER-CQL (RBS) achieves 0.98 ± 0.25, 5.80 ± 0.17, and 1.24 ± 0.12, while CAPER-CQL (TAS) achieves 1.19 ± 0.17, 5.87 ± 0.15, and 1.27 ± 0.20. These values are means and standard deviations. Both variants retain higher mean returns than BC, CQL, and CQL-Prior on all three tasks. Tables 1 and 5 report the corrected BC results and each CQL-based method's results with its best-performing configuration overall among those tested so far. Detailed settings are provided in Appendix B.3, Table 2. Our anonymous repository also includes the BC and CQL implementations.

Q4: Action projection

A4: Thank you for the question. Action projection maps generated text to an admissible environment command: “The next action should be: take tomato” becomes “take tomato” when that command is admissible. The RL policy selects among these projected candidates. We have added the definition in Section 3.1 and the example in Appendix B.3.

We sincerely appreciate the reviewer's time and thoughtful suggestions, which have helped us improve the clarity of the paper and strengthen its experimental evaluation.

For A1, my concerns are not addressed. Rather, it seems to me that the authors are switching their emphasis of the core contributions here. The central claims in the paper are still not well supported. I keep my rating for this submission.

Thank you for clarifying your concern. CAPER’s contribution remains a closed loop between RL policy learning and failure-guided action-prior refinement using external task knowledge. The latency analysis was supplementary. We clarify this through the following four points.

  1. With the same knowledge sources, refining action guidance from actual failures provides additional value. Available environment knowledge does not tell designers which rules or action preconditions the current model needs emphasized. On the TextWorld-Commonsense (TWC) task, replacing failure trajectories with the task description while retaining the knowledge base, RL backbone, prompt generator, selector, and update rule reduces returns under both RBS and TAS (Figure 8(a)). We also compare CAPER against Static Full-KB, which summarizes all knowledge into a fixed prompt before training, Direct-KB, which supplies complete knowledge at every decision, and Direct-KB + History, which adds three recent low-return trajectories. CAPER-DQN (TAS) achieves higher mean returns than all three under both comparison criteria in the table below (Figures 7, 8(b), and 19). Adding failure history does not improve Direct-KB in this setting. Together, these comparisons support using failure feedback for targeted prior refinement, rather than simply appending failure histories to action-generation inputs. Refinement remains useful when all available task knowledge is supplied from the outset. Starting from the same Direct-KB initialization, we keep the knowledge text fixed and refine supplementary action guidance. Under the same total number of environment episodes, this CAPER-DQN (TAS) variant reaches 5.82, compared with 4.92 for fixed guidance (Appendix B.15.7, Figure 21).
MethodSame RL-training episodesSame total env. episodes
Static Full-KB3.95 ± 0.503.95 ± 0.50
Direct-KB4.92 ± 0.204.92 ± 0.20
Direct-KB+History2.87 ± 0.262.87 ± 0.26
CAPER-DQN (RBS)4.99 ± 0.304.33 ± 0.40
CAPER-DQN (TAS)5.41 ± 0.315.34 ± 0.35
  1. Prior refinement increases the coverage of correct actions, addressing candidate omissions that the downstream RL selector cannot directly repair. At a given decision, the selector can choose only among the actions proposed by the LLM. Updating its Q network does not directly change which actions the LLM proposes for the same input. We isolate this effect on TWC by holding the Q network and state-history inputs fixed before and after prior refinement. Correct-action coverage rises from 71.07% to 84.83%, and overall correct selection rises from 54.40% to 68.33% (Table 12). The two updates thus serve complementary roles: prior refinement improves the available candidates, while RL learns to select among them.

  2. Continued refinement helps keep action guidance suited to the RL policy's current stage of learning. Repeated refinement outperforms keeping the first refined prompt fixed (Single Refinement) when all environment episodes are counted (Figure 15). On TWC, the original prompt previously obtained a mean evaluation return of 2.20. With the policy, initial prompt, and refinement settings fixed, feedback from observed failures yields three candidates averaging 4.80 in return, compared with 2.03 when the feedback describes failures not observed in the policy's rollouts. This supports grounding refinement in the problems the current policy actually encounters. In a complementary experiment on Overcooked (Salad), both fixed policies achieve higher success rates with prompts from their corresponding training stages across ten random seeds, as shown below. Candidate coverage measures whether correct actions are available, while task success also depends on the policy’s choices throughout a trajectory. The cross-stage results complement this analysis by indicating that effective guidance also depends on the policy–prompt pairing. Together, these results support adapting guidance to the policy as it learns.

Fixed RL policyEarlier-stage promptLater-stage prompt
Earlier-stage policy90%70%
Later-stage policy70%100%
  1. CAPER provides a reusable automatic refinement process that reduces the need to manually rewrite prompts for each task and failure. Access to task knowledge still leaves the designer to decide how to express it as action guidance and what to change after a failure. Given this knowledge and interaction feedback, CAPER diagnoses failures, retrieves relevant information, and generates candidate revisions. RBS or TAS selects the guidance used in subsequent RL training, without manual prompt editing at each round. We use the same refinement procedure across the evaluated environments and RL backbones, allowing task-specific guidance to evolve without redesigning the refinement process for each task.

Thank you for the opportunity to clarify CAPER’s contribution. We hope our additional experiments and clarifications address your concerns about closing the loop between LLM action priors and reinforcement learning, particularly the value of failure-guided refinement when the same environment knowledge is available.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.