acceptodds
Under review as a conference paper at ICLR 2027

Think in Language, Act in Code: Searching Programmatic Policies via Natural Language

Abstract

Programmatic policies offer an appealing representation for reinforcement learning because they can translate the general knowledge encoded in foundation models into functions that issue low-level actions in the environment. However, synthesizing effective programs through interaction with an environment is challenging, as the space of possible programs is vast and small syntactic changes can lead to large and often unpredictable changes in behavior. We tackle this challenge by searching in an abstraction of the program space through LLM-proposed natural language programs, using feedback from environment interactions to guide the search. Each natural language program specifies a candidate strategy and is translated into an executable policy through iterative Python code synthesis. This creates a two-level search process in which natural language guides the exploration of strategies while program synthesis grounds these strategies into executable policies. Experiments on a large set of MiniHack tasks show that searching through natural language programs improves both the efficiency of programmatic policy search and the quality of the policies discovered.

Primary area:
reinforcement learning
Keywords:
Programmatic Reinforcement Learning, Program Synthesis, LLM-Guided Search, Policy Search, Natural Language Abstractions

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_5Wz4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TACO searches for programmatic control policies using two coupled levels. An outer progressive-widening/UCB tree searches structured natural-language policies consisting of a description and staged plan. An inner process holds each plan fixed while refining executable Python code from rollout feedback. An LLM judge classifies realization as partial, full, or abandon, governing refinement termination and subsequent semantic expansion. A shared language knowledge library transfers execution-derived facts across branches. The paper evaluates five MiniHack tasks with the full 121-action NLE space, compares against MLES adapted to UCB search and direct generative code optimization, and analyzes refinement, representation, judging, and knowledge.

Critical strengths
  1. The distinction between a strategy and its code realization addresses a real problem in LLM-guided policy search: a useful strategy can be discarded because its first implementation is defective. Committing to multiple execution-guided refinements while holding the NLP fixed is a coherent response. The realization judge also distinguishes following a plan from obtaining a high task return, allowing a faithfully implemented but poor strategy to inform later exploration. This is more substantive than merely attaching a natural-language explanation to each generated program.

  2. The empirical study includes relevant baselines, repeated search runs, and two search-cost measures. Figure 2 reports substantial gains on WoD, Quest-Easy, and KeyRoom, with competitive performance on the two corridor tasks. The experiments use 10 independent runs on the first three tasks and 5 on the corridor tasks, and count all generated output tokens, including judge and knowledge outputs. MLES-UCB controls the broad search mechanism, while GCO provides a direct-code refinement comparison. The structured-plan and judge ablations further support the usefulness of the overall design.

Critical weaknesses
  1. The reported best discovery score is not an independent estimate of the selected program's expected performance. Figures 2, 4, and 5 use the best score observed during search. Appendix B, Tables 2-4, evaluates each Quest-Easy candidate on only five episodes and candidates elsewhere on 20. No separate reevaluation of the selected programs on fresh environment seeds is specified. Selecting the maximum among many noisy estimates can make both final quality and discovery speed optimistic, and can affect methods differently when their search paths expose different candidates. For example, the score of 1 defined at p5, lines263-264 can mean 5/5 successes on search evaluation episodes; independent reevaluation is needed to support a robust task-solved conclusion. The independent search runs and standard-deviation bands are valuable, but do not remove this selection effect. This leaves a central claimed benefit--the quality of the discovered policies--insufficiently validated.

  2. The specification does not consistently establish that a node's retained code realizes the strategy whose value the outer search is estimating. The formulation defines V(psi) over programs realizing psi (p3), but Section 3.3 retains the highest-return program while describing the node verdict as the terminating verdict of the current realization (p5, lines231-246). Refinement may reduce return, so the best retained code and current code can differ. Figure 1 instead says the node inherits its best policy's verdict. Which rule is implemented matters: a full verdict could otherwise cause expansion to preserve stages not realized by the inherited best code. Appendix C.5 treats earlier stages as confirmed during stagewise judging, but it is unclear whether confirmations reset for every new code candidate and all stages are rechecked. Figure 3 measures the same judge's full labels, without an independent fidelity audit. These gaps weaken the interpretation that the method separates strategy quality from implementation quality, even if the discovered executable policies themselves remain useful.

Questions for authors
  1. Can the frozen selected programs be reevaluated at final or representative matched-budget checkpoints on a substantial fresh set of environment seeds, including fresh layouts where the task already randomizes them, without further updates and with uncertainty estimates? Please specify whether search evaluations reuse reset seeds and whether test rollouts were ever supplied to the optimizer. Do the three hard-task gains and task-solved conclusions persist? This requests validation of the existing discovered policies, not success on a new benchmark suite.

  2. What verdict/routing rule replaces the judge in Figure 5? Since verdicts also select Refine versus Discover expansion and abandonment, the comparison does not isolate adaptive stopping unless this routing is controlled. Are knowledge updates from the judge's new_knowledge field used, and are they held comparable across variants? A fixed-refinement control with terminal judging and matched routing would distinguish these effects.

3 comments

We thank the reviewer for the thorough and detailed comments. Below we address and answer all the questions. We believe addressing them improved the clarity and soundness of our work.

W1/Q1) Held-out test performance.

We appreciate the Reviewer's comments and agree that independent evaluation provides a more reliable measure of the quality of discovered programs. In the revised version, we have updated the evaluation protocol (Section 4). In particular, during search, we evaluate each candidate program on a fixed number of episodes, using a fresh random environment seed for every episode. Search seeds are never reused, and test rollouts are never exposed to the optimizer. Figure 2, 4, 5, and the ones we added during the rebuttal now report the held-out return performance. At each checkpoint, the selected program is evaluated, without further modification, on a fixed set of 100 held-out test seeds disjoint from search seeds and shared across methods, including fresh layouts where applicable. Despite this change, the considerations and conclusions about our proposed method remain unchanged (Appendix H).

W2) Misalignment between retained code and value estimate. We thank the Reviewer for pointing this out! We revised Figure 1 accordingly. In our implementation, a node's verdict is determined by its final realization, whereas its value and the program passed to its children are determined by its best-return realization. This design choice serves two complementary purposes: (i) the value guides the outer search toward promising branches as a practical proxy for V(ψ)V(\psi), (ii) the verdict determines whether expansion should refine the current plan or discover new stages.

A disagreement arises only when an earlier realization outperforms the final realization judged as fully implemented, as ties favor the newer program. This scenario is very rare and only occurs in 4% of fully implemented nodes across all runs and environments. If the judge’s verdict was correct, retaining the higher-return program preserves useful behavior that may not yet be captured by the realized plan, providing an opportunity for further improvement. On the other hand, if the judge incorrectly marks an NLP as “fully implemented,” a later expansion can correct the error with refinements over its stages.

About stagewise judging. At each refinement step tt, the judge produces a verdict which includes re-verifying stages inherited from a parent (Section 3.3).

About judge fidelity. To assess the reliability of the realization judge, we independently re-labeled the verdicts underlying Figure 3 using Claude Sonnet 5.5 (Appendix I). Our judge (Gemini-3.1-Flash-Lite) agrees with Claude Sonnet on 85% of verdicts, although our judge assigns full realization somewhat more frequently (Appendix I, Figure 11). Additionally, we re-compute the results of Figure 3 while matching the agreement between our judge and Claude Sonnet (Appendix I, Figure 10). This new analysis yields lower absolute realization rates but preserves the same trend: the proportion of fully realized NLPs increases with additional refinement steps. This supports the importance of allowing multiple refinement steps rather than relying on a single realization attempt.

Q2) Judge ablation (Figure 5).

We thank the Reviewer for the insightful comment, and following the Reviewer’s suggestion, we revised the manuscript and extended our ablation study (Section 3.3, Section 5.2 and Figure 5).

About the judge's new_knowledge field. There are two ways we add information to the knowledge: one way is that at each refinement step, LLMknow\text{LLM}_\text{know} will add to it, and the second way is that the judge can also add knowledge during its verification process. We modified Section 3.3 accordingly.

Improved Figure 5. From the submission’s version, we kept the return-based (previously referred as without judge) variant that assumes a fully implemented verdict when the refined program improves over the parent's program and is otherwise partial. This variant never returns the verdict abandon. Additionally, we added two new variants: terminal judge and adaptive judge. The former has a fixed number of refinement steps and only judges the terminal refined program with the three possible verdicts. This way, it will match the expansion routing of TACO. The latter judges at every refinement step, and a fully implemented or abandoned verdict stops the refinement early. All these variants do not modify the knowledge and only the LLMknow\text{LLM}_\text{know} will do so, if needed. When comparing these variations with TACO, we can see that the main benefit from the judge is the adaptiveness. We modified Figure 5 to include all these variants.


We hope these clarifications address the reviewer's concerns and we would be happy to answer further questions during the discussion period.

Thank you for the substantial revisions. The held-out evaluation and terminal/adaptive judge controls address my main concerns. Please clarify whether the code/verdict mismatches, reported in 4% of fully implemented nodes, affect successful search paths or final selected programs. The 85% agreement with a stronger LLM provides useful consistency evidence, but should not be interpreted as ground-truth validation of realization fidelity. These limitations warrant qualified mechanism claims. Overall, the expanded empirical evidence supports increasing my recommendation.

AuthorsOpenReview ↗

Clarification

We thank the reviewer for the positive reassessment.

Code/verdict mismatches. We traced all mismatched nodes through the search trees across all runs. None of these nodes appear on a search path leading to a final selected program. Thus, these mismatches do not affect the final selected programs or their reported performance.

Judge fidelity. We agree that agreement with a stronger LLM provides evidence of consistency, but does not establish ground-truth realization fidelity. We will clarify this limitation in the paper and qualify our claims accordingly.


We hope these clarifications address your remaining concerns. If so, we would appreciate it if you could consider updating your recommendation.

Reviewer_v4KnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

Programmatic reinforcement learning uses programs as policies, providing a way to distill knowledge encoded in LLMs into policies. While previous works focus on using LLMs to evolve the program directly, this paper proposes Think in Language, Act in COde, where it instead evolves the strategies and then translates them into executable programs. It uses a tree-like structure, where each node represents a Natural Language Policy (NLP) containing the plans that can solve the task potentially. Each plan is then translated into executable programs and is iteratively rewritten to ensure that the programs follow the plan. Each node (NLP) is sampled using a UCB-style selection rule, where the search value is determined by the best returns among the programs. Experimental results on 5 MiniHack show that their proposed framework outperforms previous baselines, while ablation studies verify the necessity of each component.

Critical strengths
  • The paper is generally well-written and easy to follow.
  • The proposed "Committed code realization" is interesting and shows promising improvement by reducing the randomness induced when translating natural language into programs.
Critical weaknesses
  • The proposed method is only evaluated on one environment; evaluating TACO on other environments, such as Atari, as in GCO, can make the results more pronounced.
  • As shown in Figure 3, different LLMs exhibit different behaviors when increasing the refinement steps. The authors should verify that TACO outperforms other baselines across different LLMs.
  • The condition under which the node is considered "abandon" is not explained.
  • The details of the compared baseline MLES-UCB are missing, and the reason for not comparing to the original MLES is not convincing enough.
Questions for authors
  • The authors state in the Search configuration that they use the same hyperparameters across environments, but Table 2 in Appendix B shows varying values. Which one is correct? Are the results sensitive to hyperparameters?
  • The authors state that they normalize the return using the range of returns observed during the search; does this mean that RminR_{min} and RmaxR_{max} change during the search? Will it make QAQ_A unstable?
4 comments
AuthorsOpenReview ↗

Response Part 1

We thank the Reviewer for the extensive and detailed feedback on our manuscript. Below, we address each of your comments and questions. We found them extremely helpful in further improving the quality of our manuscript.

W1) Adding more environments.

We thank the Reviewer for the comment and for raising an important question regarding a broader evaluation of our method. In the revised version of the work, we have added two additional domains: Montezuma's Revenge and MicroRTS [1] (Appendix D).

Montezuma’s Revenge. This domain presents a difficult exploration challenge, as securing the key for the initial reward demands an extensive, precise sequence of steps. To manage computational expenses while maintaining the core search difficulty, we cap episodes at 150 environment steps across all baselines. Because object-centric states omit certain visual elements, we augment both TACO’s and baselines’ text trajectories with episode descriptions extracted from rendered images via a visual perception model (Gemini-3.1-Flash-Lite). TACO surpasses both comparative baselines (GCO and MLES-UCB) in mean held-out test performance, while requiring significantly fewer environment steps and LLM sampling tokens (detailed metrics are provided in Appendix D.1, Figure 6).

MicroRTS. MicroRTS is a real-time strategy game in which players simultaneously control units, gather resources, construct buildings, and engage in combat. Winning requires coordinating resource management, unit production, and combat tactics to destroy the opponent’s units and buildings. The task is to find a best-response program that defeats MayariBot, a strong scripted opponent. We conduct five independent runs per method with a budget of 1 USD per run. TACO finds a program that defeats MayariBot in all five runs, compared with three of five for MLES-UCB and zero of five for GCO, while evaluating about half as many programs as MLES-UCB. Table 6 in Appendix D.2 reports the per-run results.

W2) TACO and baselines performance across different LLMs.

We thank the Reviewer for pointing out this important additional study. In the revised version, we ran TACO, MLES-UCB, and GCO with two additional LLMs: Qwen3.5-122B-A10B and Gemini 3 Flash. The former is generally weaker than the one we used (DeepSeek-V4-Flash), while the latter provides a stronger backbone. We compare the results across two different environments: Wand-of-Death-Hard (WoD) and Quest-Easy (Appendix E; Figures 7 and 8). The results reveal three regimes depending on the LLM’s ability to solve the task. At one extreme, Gemini 3 Flash can solve WoD through direct code generation, and the baselines match TACO’s final return. At the other extreme, Qwen3.5-122B-A10B fails to propose a strategy that yields positive reward on Quest-Easy, and none of the methods make progress. Between these extremes, the LLM proposes promising but incomplete strategies, and TACO consistently outperforms the baselines by refining those strategies into more effective programs. This suggests that TACO is particularly beneficial when the LLM can identify useful strategies but struggles to implement them reliably through direct code refinement.

W3) The condition under which the node is considered "abandon" is not explained.

We thank the Reviewer for the comment. We agree that the paper lacked a clear explanation. In the revised version, we added additional details in Section 3.3. The judge returns abandon only when the code follows the plan, but the environment makes a step impossible regardless of the code, e.g., an unreachable location or a missing item.

W4) MLES-UCB missing details and MLES discussion.

We thank the reviewer for the insightful comment. We introduced MLES-UCB to compare TACO with MLES [2] while controlling for the choice of search algorithm. Using the same progressive-widening UCB tree search in both methods allows us to assess whether TACO’s gains persist when the search algorithm is held fixed. This comparison follows the design of MLES, which is search-agnostic and presents evolutionary search as one instantiation (Hu et al., 2026, Section 2.4.1). In the revised version, we added a direct comparison between the original MLES and MLES-UCB (Appendix F, Figure 9). The original MLES fails to make progress on Quest-Easy, where MLES-UCB substantially improves performance, and the two variants perform comparably on Corridor-R3. We then added two evolutionary baselines, ShinkaEvolve [3] and OpenEvolve [4], and observed the same pattern across all three evolutionary methods. They struggle most on tasks where finding an initial program that earns positive reward is difficult. These results support using MLES-UCB as a stronger baseline for TACO, since it improves on the original MLES on Quest-Easy and matches its performance on Corridor-R3.

AuthorsOpenReview ↗

Response Part 2

Q1) About hyperparameters configuration during search and sensibility.

We thank the Reviewer for pointing out this issue. In the revised version, we corrected the text (Section 4) to reflect the correct narrative. Additionally, following the Reviewer’s suggestion, we studied the sensibility of our method to different values of its core hyperparameters: commitment budget MM and progressive-widening coefficient kk.

We compare TACO’s performance across values of MM (Figure 5, left). A budget that is too small leaves some strategies unrealized, while an overly large budget spends resources on unpromising strategies. Using the same experimental setup, we examine how kk shapes the search. Smaller values of kk lead the search to pursue fewer strategies in greater depth, while larger values broaden the range of strategies it tries. We used this trade-off to guide our choice of kk within the range considered, without an extensive hyperparameter sweep. The results also suggest that incorporating knowledge makes the search less sensitive to the choice of kk (Appendix G, Table 7).

Q2) About return normalization. Does this mean that RminR_{min} and RmaxR_{max} change during the search? Will it make QAQ_A unstable?

We thank the reviewer for the question. RminR_{min} and RmaxR_{max} can change during the search because they track the minimum and maximum returns observed among resolved nodes. When Rmin=RmaxR_{min}=R_{max}, the normalization denominator is zero, so we assign the newly resolved node a normalized return of 0.50.5. We clarified this in Section 3.2.

The effect of running normalization on QAQ_A is bounded. Each node’s normalized return is computed once, when it resolves, so a change in the range leaves previously assigned values unchanged. When the range changes once, the difference in QAQ_A relative to recomputing those values under the updated range is max⁡(0.5,mA/nA)\max(0.5,m_A/n_A), where mAm_A is the number of nodes contributing to QAQ_A that resolved before the change and nAn_A is the total number of nodes contributing to QAQ_A.

Across our experiments, the range never changed more than once per run. We also compared running normalization with a fixed range of [0,1][0,1] on Quest-Easy. Both variants solved all five seeds, requiring 23.0±3.423.0 \pm 3.4 policy updates with running normalization and 25.0±5.125.0 \pm 5.1 with the fixed range. We observed similar performance in this five-seed comparison.


We believe we have addressed the reviewer's concerns and would be happy to answer further questions during the discussion period.

[1] Santiago Ontañón. The Combinatorial Multi-Armed Bandit Problem and its Application to Real-Time Strategy Games, In AIIDE 2013. pp. 58 - 64.

[2] Hu, Qinglong, et al. "Multimodal LLM-assisted evolutionary search for programmatic control policies." International Conference on Learning Representations. Vol. 2026. 2026.

[3] Lange, Robert Tjarko, et al. "ShinkaEvolve: Towards open-ended and sample-efficient program evolution." arXiv preprint arXiv:2509.19349. 2025.

[4] Sharma, Asankhaya. "OpenEvolve: An open-source evolutionary coding agent." GitHub. 2025. https://github.com/algorithmicsuperintelligence/openevolve

I thank the authors for their thorough responses and the extensive experimental results. Since all my concerns and questions have been resolved, I'm happy to raise my score to 3.

We are glad our rebuttal addressed all concerns and thank the Reviewer for revising their score.

Reviewer_MRGROpenReview ↗
Rating
4(+3)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces an LLM orchestration method to produce programmatic policies for navigation-like tasks in game environments. The orchestration consists of a higher-level natural language plan evolution along with an iterative programmatic implementation of each plan, i.e., multiple attempts are made at generating a programmatic policy from a natural language plan. The natural language plans are semantically structured and the search in the plan space is also guided in a specific way to make small progressive changes. This search uses progressive widening and also UCB on the tree of plans is used to pick plans for policy generation. A knowledge bank is also maintained which is updated with useful environment facts learned during execution of policies. The authors implement this and compare against two other similar baselines, showing that their method is able to learn to perform well in the environments with less environment interaction.

Critical strengths
  1. Unique methodology with some interesting components: The architecture presented by the authors is interesting. They use a flexible domain-specific semantic structure for the natural language plans which enables principled exploration. This is an idea that can be transferable to other search tasks.

  2. Complete ablations on the architecture: The ablations presented by the authors are thorough and show the utility of each component in their method.

  3. Well written and easy to understand

Critical weaknesses
  1. Lack of important baselines: My main concern with this work is the lack of evaluation of simpler harnesses. This is a generation task with a clear fitness function given by the rollout reward. Thus, any search harness can be used. A simple baseline is the Ralph loop [1] where an LLM repeatedly makes an attempt while storing any learnings in a file (can be used through OpenCode [4]). Also, OpenEvolve [2] and ShinkaEvolve [3] are relevant evolutionary search baselines that should be considered. I believe that these baseline comparisons are important because recent work has shown that simple orchestration methods can often outperform complex methods [5].

[1] https://github.com/snarktank/ralph

[2] https://github.com/algorithmicsuperintelligence/openevolve

[3] https://github.com/SakanaAI/ShinkaEvolve

[4] https://github.com/anomalyco/opencode/

[5] Automated Discovery Has No Universally Superior Harness. Gupta et al. https://arxiv.org/abs/2607.18235

Questions for authors
  1. I request the authors to address the above described weakness regarding insufficient baseline comparisons. I am happy to raise my score to an accept if the authors provide further baseline comparisons or if not, justify why they are not relevant.

  2. Clarification: what feedback does the method get after interacting with the environment? Does it receive a full rollout trace along with the score?

3 comments

We thank the Reviewer for the thoughtful and constructive comments. Below, we explain how we addressed them. We believe that addressing your feedback has improved the quality of our paper.

W1/Q1) Lack of important baselines.

We thank the Reviewer for suggesting that we compare with other search harnesses. Following the Reviewer’s suggestion, we have added to our experimental analysis all the suggested baselines: Ralph loop [1], OpenEvolve [2] and ShinkaEvolve [3].

All the methods follow the official implementation and they share the same LLM (DeepSeek-V4-Flash), task description, initial policy, feedback with mean return as fitness and no internet access. Additional details about the training, evaluation setup and costs are reported in Appendices B and C. In the revised version, we updated Figure 2 and Section 4 to include the performance comparison of all the baselines. Moreover, in Figure 2 we added a third row of plots where we compare the LLM input tokens. Next, we summarize the results.

Evolutionary baselines need some reward. All environments have sparse rewards, so evolutionary selection only has a signal once the LLM proposes a program that earns some reward. On Corridor-R3, CorridorBattle, and KeyRoom this happens easily, and OpenEvolve and ShinkaEvolve make progress. On WoD and Quest-Easy, almost no candidate achieves any positive reward, and performance stays near 0 despite 6-20x more interactions than TACO.

Ralph is the strongest baseline but far less efficient. TACO still outperforms RALPH on four of the minihack environments and they compete very closely on the CorridorBattle. However, it needs about 3.4x more interactions and 17x higher cost ($) on average, mostly from input tokens 19-73 M/run (Ralph) vs. 0.25-2.4 M/run (TACO).

In Reinforcement Learning sample efficiency is a key metric as environment interactions are costly. Across all environments TACO reaches its final performance with fewer interactions than Ralph, OpenEvolve, and ShinkaEvolve in every environment.

Q2) What feedback does the method get after interacting with the environment?

Each method, after interacting with the environment, receives as feedback a full rollout trace along with the returns. Appendix J.2 provides a clear and detailed example of the feedback. All baselines receive exactly the same feedback.


We hope these additions address the Reviewer's concern, and we would be happy to answer further questions in the discussion period.

[1] https://github.com/snarktank/ralph

[2] https://github.com/algorithmicsuperintelligence/openevolve

[3] https://github.com/SakanaAI/ShinkaEvolve

I thank the authors for their response. I am satisfied with their addition of important baseline comparisons and so I am raising my score to 4.

We thank the Reviewer for considering our rebuttal and updating their evaluation of our manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response Summary

We thank all Reviewers for their thoughtful feedback. We are encouraged that the Reviewers found that separating a policy's strategy from its code realization addresses a real problem in LLM-guided policy search (Rev. 5Wz4), the structured natural-language plans enable principled exploration and could transfer to other search tasks (Rev. MRGR), committed code realization is a coherent and promising idea (Revs. v4Kn, 5Wz4), the ablations are thorough (Revs. MRGR, 5Wz4), and the paper is clearly written (Revs. MRGR, v4Kn).

We have uploaded a revised version of the paper. All changes in the revised version are highlighted: new passages appear on a yellow background, and edited text is shown in blue. Below we summarize the main revisions; detailed answers are given in the individual responses.

Additional Experiment and Baselines:

  • Additional baselines [Reviewer MRGR]. We compare against three general-purpose LLM search harnesses, Ralph, OpenEvolve, and ShinkaEvolve, on all five MiniHack environments, under the same LLM, task description, and feedback (Section 4; Figure 2; Appendix B; Appendix C).
  • New domains [Reviewer v4Kn]. We evaluate TACO on two additional environments: Montezuma's Revenge and on MicroRTS, a real-time strategy game (Appendix D).
  • Different LLMs backbone [Reviewer v4Kn]. We compare TACO and baselines leveraging two additional LLM families, Qwen3.5-122B-A10B and Gemini 3 Flash (Appendix E).
  • Comparison with MLES and MLES-UCB [Reviewer v4Kn]. We analyze in detail the main differences between MLES and MLES-UCB and compare their performance against TACO (Appendix F).
  • Study on hyperparameters sensibility [Reviewer v4Kn]. We report an additional experiment testing the sensitivity of the key hyperparameters: commitment budget MM and progressive-widening coefficient kk (Figure 5; Appendix G).
  • Independent judge audit [Reviewer 5Wz4]. We assess the reliability of our judge (Gemini-3.1-Flash-Lite) comparing its verdicts with the ones of a stronger model, Claude Sonnet 5.5 (Appendix I).

Revision of the Paper:

  • Sound evaluation procedure [Reviewer 5Wz4]. We now evaluate the selected programs on 100 held-out test seeds per environment and describe the evaluation protocol in more detail (Section 4). We also report a comparison between search and test performance (Appendix H).
  • Improved ablation on Adaptive Realization [Reviewer 5Wz4]. We improved the ablation study by adding two additional variants (Section 5.2; Figure 5).
  • Improved presentation and corrected inconsistencies [Reviewer MRGR, v4Kn, 5Wz4].

We would like to thank the Reviewers again for their valuable and constructive feedback. We have done our best to provide thorough and detailed responses to the Reviewers’ comments. We hope that these revisions address all concerns, while further emphasizing the importance and robustness of our contribution. We would be happy to answer any further questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.