acceptodds
Under review as a conference paper at ICLR 2027

Neuro-Symbolic Computer Use: Learning Reusable Policies for Reliable and Efficient Execution

Abstract

Many computer tasks recur: the same workflow runs many times, with new inputs and from different starting states. Current computer-use agents re-plan every step of every run, which makes them costly and unreliable on such tasks. We introduce *neuro-symbolic computer use*, in which a recurring workflow is executed by a learned policy rather than re-derived by an agent on each run. The policy fixes the decisions that are stable across runs (ordering, variables, loops, and branches) in executable code, and delegates observation-dependent decisions, such as grounding and state checks, to neural models. We learn these policies with *neuro-symbolic policy iteration*: starting from one agent trajectory, it executes the policy, diagnoses failures with task-completion and step-level judges, and revises the code with a coding model informed by an agent's continuation from the point of failure, without access to the benchmark evaluator. Iterating on generated parameter and initial-state variants makes the policy reusable, and a pre-action verifier guards each state-mutating step at deployment. On OSWorld-Verified and ScienceBoard, the learned policies achieve the highest Pass^3 of all methods in all four settings, 3.6-15.8 points above the base agent, while cutting per-run cost by 15-217 and latency by 3.4-5.1. On OSWorld-Verified, policies built only on variants transfer to the held-out original tasks, exceeding AutoRPA by 8.6-17.5 points in Pass^3.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Computer-use agents, neuro-symbolic planning, LLM agents, agent reliability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7epFOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The system translates agent execution trajectories into reusable programs, where code manages the stable routines and neural models manage GUI-grounding and state-checking. The NSPI uses task-completion and step-level judgments, agent execution following failed roll-outs, and code repair to incrementally improve the programs. The experiments investigate the reliability of the repeated execution and number of executions necessary to pay off the construction costs on OSWorld-Verified and ScienceBoard. They also investigate the generalization capability from one instance of the workflow to another within OSWorld task families. Other experiments evaluate the pre-action verification and longer workflows on OSWorld-V2.

Critical strengths
  • Evaluation deals with practical needs of recurring workflows. In four benchmark-model pairs, NSPI enhances Pass^3 over the baseline agent by 3.6 to 15.8 percentage points. Its slight disadvantage in terms of Pass^1 in both OSWorld settings is presented as well. I appreciate such differentiation of single-run performance and stability in repetitive runs (Table 1).

  • Expenses and failure cases are explicitly mentioned. Table 3 presents construction and execution expenses; monetary cost-break-even points are about 3 to 27 runs. Construction failure stays in the denominator. Failure recovery, transfer between workflow instances, and decreased Pass^1 rate in OSWorld-V2 reveal areas of application of this technique.

Critical weaknesses
  • Contribution of individual components and budget comparability is not clear yet. Batch2Code enables iterative improvement overall but not for ablation study of dual judges and agent continuity; judge correctness in Section 4.7 cannot serve this purpose. Moreover, in A.3 the number of steps is equal for rollout of code and the baseline agent but Table 8 mentions the limit of 5×/16× for ScienceBoard. The scope and units of measurement of their budgets have to be clarified before budget equality assessment.

  • Benefits of full deployment are not yet verified. In Section 4.6, two verifiers miss 17/24 and 6/24 actions which require blockage and blockage of the correct workflow. The evaluation includes intention comments and changes in rubric. It requires end-to-end success, cost, and latency with verification, false blockage, and fallback before extending the efficiency claims to deployment under guard.

Questions for authors
  • Are the 5×/16× bounds from Table 8 applicable to final evaluation? In the affirmative, please give the matched-budget figures; otherwise, resolve the A.3. Can you also prune each judge and agent continuations given the matched budget construction?

  • Is the verification process included in the primary execution costs? What calibration data was used for θ=0.78, and what data was used for the rubric revision? Overall success, cost, latency, and break-even counts, including comments generation, verification, false blocks, and fallback.

  • The variation generator takes the evaluation scripts into account, and the families chosen should have at least one base-agent success. How do these restrictions affect the generalizability of the findings? Please list the variants and confidence intervals, as estimated at the task family level. The interpretation of the AutoRPA results is, thus, a comparison of these particular workflow variants.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7epF comments (Part 1)

We thank the reviewer for the insightful comments and questions. We have addressed all your comments below.


How are execution budgets matched, and what does each refinement component contribute? (W1/W2/Q1)

On ScienceBoard, we use the 5×/16× limits to match the code replay budget to the base agent’s execution budget. ScienceBoard limits the base agent in planning calls, and each call can issue several primitive actions. For a task allowing n planning calls, we cap each code rollout at 5 × n state-changing primitive actions for Terra and 16 × n for Opus 5 based on inspecting the model baseline runs (Table 8). The same replay cap applies during refinement and final evaluation. We will clarify the two units in Appendix A.3 and Table 8.

Contribution of each component. We conducted component-wise ablations on 60 OSWorld tasks with Opus 5. Starting from the initial neuro-symbolic policy, we measured the gains from adding judge-guided refinement and then agent continuation during refinement. Both components improve reliability, with judge-guided refinement providing the larger Pass^3 gain. The optional pre-action verifier was disabled for the main results, and we discuss its deployment role and costs below.

ConfigurationPass^3 (%)Upfront cost ($/task)
Base agent73.3—
Initial neuro-symbolic policy50.00.10
+ judge-guided refinement73.34.46
+ agent continuation during refinement (full NSPI)78.34.89

What does pre-action verification add at deployment, and what do the reported costs include? (W3/Q2)

The role of the pre-action verifier in practical deployment. We introduced the pre-action verifier as an optional check for practitioners who need to guard a neuro-symbolic policy against drifts in environment dynamics. It can block a state-changing action if the current screen no longer matches the action’s preconditions and hand control to an agent or a human. Section 4.6 evaluates its accuracy separately in an offline setting because it is a separate deploy-time decision for practitioners to make. We ran all main results, including the reported costs, with verification disabled because OSWorld and ScienceBoard do not exhibit environment nonstationarity, and we will make that distinction explicit in the manuscript. The θ=0.78\theta = 0.78 was calibrated by sweeping θ\theta for wrong-block rate ≤5%\leq 5\% on 387 human-labeled steps from out-of-distribution agent runs: OSWorld-style Chrome tasks on live webpages on a Windows operating system. We also utilize this offline set for informed rubric revisions.

What does a verifier block imply for performance and cost? A verify pass lets the policy execute the same action it would take during ordinary replay. A block hands the remaining task to the base agent or a human. After an agent handoff, success depends on whether the agent can complete the task from the current state. Useful progress made before the block can reduce the remaining agent work. Guarded replay retains a per-run cost advantage when the policy steps already executed, verifier calls, and any fallback together cost less on average than running the base agent from the start.

AuthorsOpenReview ↗

Response to Reviewer 7epF comments (Part 2)

How do variant generation and family selection affect the generalization findings? (Q3)

Evaluation-script access. The variant generator receives the original evaluation script to keep each variant’s specification and evaluator consistent, but this doesn’t impact generalization results as both methods don't use the evaluation script during exploration and refinement. After construction on the generated variants, the policies are frozen and tested on the held-out original task using its benchmark evaluator.

Task-family selection. We focus on families whose original benchmark task was solved at least once by a base agent to help disentangle policy generalization from underlying task difficulty. If the base agent cannot solve the original task, failures of policies seeded from that agent may reflect its limitations rather than failures to transfer across variants. This setup also accommodates AutoRPA’s reliance on a successful seed trajectory, whereas a successful seed is not required for NSPI. Both methods conduct their own exploration on the generated variants and are not given the successful original-task trajectory.

Task-family-level confidence intervals. NSPI’s advantage is statistically supported on parameter variations, while the initial-state estimate favors NSPI by 8.6 percentage points but has greater uncertainty (95% CI: −8.6 to 25.7). Motivated by the judge-annotation analysis in Section 4.7, we also repeated policy construction after a simple change to the step judge’s inputs: providing program return values. This raises Pass^3 from 55.0% to 57.5% on parameter variations and from 60.0% to 71.4% on initial-state variations. We will add the full variant list and confidence-interval estimation details to the appendix.

Variation (families)AutoRPAOriginal NSPIDifference [95% CI]NSPI (judge with return values)Difference [95% CI]
Parameter (40)37.555.0+17.5 [5.0, 32.5]57.5+20.0 [5.0, 35.0]
Initial state (35)51.460.0+8.6 [−8.6, 25.7]71.4+20.0 [0.0, 40.0]
Reviewer_wKNrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces 'neuro-symbolic computer use' as a paradigm where the policy is a program with access to 'symbolic primitives' that do not require use of (parametric) models; and 'neural primitives' make use of parametric models through an API.

The paper also discusses how it refines the policy (in Sec 3). Using a method inspired by MDP policy iteration, they evaluate rollouts under LLM-backed judges, and improves the current policy with another 'healer' LLM.

As for experiments, the paper evaluates on OSWorld-Verified and ScienceBoard. The baselines compared against are base agent (no conversion), Batch2Code (conversion but no refinement), and ASI. Table 1 shows that NSPI either outperforms the best baselines or compares comparatively, while being much cheaper than either Base agent or ASI.

Critical strengths
  • policy refinement heuristics in Sec 3 is easy to understand
Critical weaknesses
  • learning framing is a bit confusing as no parametric model learning is happening: conventionally learning under neuro-symbolic models means learning the neural network parameters under some objective; but here the parameters are strings and it is not clear what is the objective. Reframing this work as LLM harness may make this work more approachable.
  • related to the previous point, not only there is not a clear objective being optimized, the paper claims the learning method (Sec 3.1.1) is policy iteration which seems incorrect. While the proposed method is intuitive, there does not seem to be any formal guarantees (in contrast to the MDP policy iteration method).
Questions for authors
  • Can you discuss theoretical aspects of the policy refinement method? Are there formal guarantees under reasonable assumptions?
  • Can you discuss / experiment with baselines that learn neural network parameters, perhaps on open-weight models?
7 comments
AuthorsOpenReview ↗

Response to Reviewer wKNr comments (Part 1)

Thank you for your time and we greatly appreciate the thoughtful comments! We have addressed them below.


What is learned and toward what objective? (W1, W2)

NSPI learns a reusable neuro-symbolic policy, represented as an executable program that combines symbolic operations with calls to neural models [1]. Mapping NSPI to Mitchell’s definition of learning [2], the task (T) is to execute that workflow repeatedly as its inputs and initial state vary. The experience (E) is a seed trajectory followed by executions of the current program, the judges’ verdicts and failure diagnoses, and the agent’s attempts to complete a failed run. The performance measure (P) is expected Pass^k over the workflow family (Eq. 1), where Pass^k is the probability that all k independent executions succeed on a given instance (Eq. 2). We separately report amortized cost per execution, which spreads the one-time cost of constructing the program over repeated runs (Eq. 3). Our approach also optimizes cost via the design choice of using neuro-symbolic programs as the policy representation, and our results show it is highly effective compared to baseline agents (Tables 1 and 3). Further optimizing cost during the NSPI iteration is possible but not the focus of this work, NSPI provides the core algorithmic structure for subsequent work to do further cost optimization. Thus, NSPI learns a reusable neuro-symbolic policy from execution experience, with expected Pass^k as its performance measure.

Learning from experience with fixed model weights has an established precedent in in-context learning, where examples in the prompt guide a model’s behavior [3]. In NSPI, what persists across runs is the revised program itself. For example, the initial program in Fig. 2 changes the background of only the first slide. After the judge identifies the failure, the code healer incorporates the agent’s edit to all slides and adds a conditional reload. Subsequent runs execute that revised procedure. To make this distinction clearer, we will describe NSPI as a test-time iterative method for learning reusable programs from execution feedback. It can refine a policy derived from a trajectory produced by any compatible agent or harness, even when that trajectory only partially completes the task.


Policy-iteration inspiration and empirical improvement (W2, Q1)

NSPI takes conceptual inspiration from Sutton and Barto’s description of generalized policy iteration (GPI) (Section 4.6) [4], in which evaluation of the current policy guides its improvement. We do not claim that NSPI implements the classical policy-iteration algorithm or inherits its theoretical guarantees. Our wording ‘by policy iteration’ (Section 3.1.1) suggests a closer correspondence than we intended. We will revise the manuscript to clarify how GPI motivates our refinement procedure.

The judge and code repair steps of NSPI correspond to policy evaluation and policy improvement in GPI. Policy evaluation: in GPI, policy evaluation updates the current policy's value function, which ranks actions at each state. In NSPI, the judge's verdict and diagnosis serve as an implicit ranking of the current policy's selected action at a program state and provide information about better actions. For example, if the step-level judge identifies that an unexpected popup blocked a click, for an LLM trained with sufficient web knowledge and coding, its diagnosis can suggest closing the popup before retrying the click. Policy improvement: in GPI, the policy is updated to be greedy with respect to the value function. In NSPI, the code healer uses the judges’ diagnosis and the agent’s continuation to propose an edit to the executable policy.

Using an LLM to propose policy edits follows a broader precedent for adapting optimization procedures to use LLMs for gradient-free updates. Yang et al. [5] prompt an LLM with previous candidate solutions and their objective values to propose the next candidate. In the policy-iteration setting, Brooks et al. [6] use in-context rollouts to estimate action values and select the highest-valued action. In NSPI, the proposed update is an edit to the executable policy.

An incorrect diagnosis or an edit that fixes one case while breaking another can lower Pass^k, so classical guarantees of monotone improvement and convergence do not carry over. In our best-so-far analysis, retaining the best policy seen so far for each task does guarantee that judge acceptance (i.e., the fraction of tasks whose selected policy the task-completion judge accepts) is nondecreasing by construction. To assess whether refinement also improves execution reliability, we measure Pass^k across iterations using the held-out benchmark evaluator.

AuthorsOpenReview ↗

Response to Reviewer wKNr comments (Part 2)

Does refinement improve the policy? Across iteration 1 to 5, held-out benchmark Pass^3 improves in all four settings, with gains of 8.3–25.2 percentage points. As the table shows, Pass^3 rises or stays flat at every iteration in three of the four settings. On ScienceBoard with Opus 5, Pass^3 peaks at 47.9% at iteration 3 and finishes at 45.0%, still 21.9 points above iteration 1.

SettingMetricIter. 1Iter. 2Iter. 3Iter. 4Iter. 5
OSWorld · TerraPass^344.353.554.856.557.3
Judge acceptance33.852.457.960.963.2
OSWorld · Opus 5Pass^347.465.168.769.072.6
Judge acceptance41.366.570.471.571.7
ScienceBoard · TerraPass^320.120.120.721.928.4
Judge acceptance16.030.835.537.938.5
ScienceBoard · Opus 5Pass^323.139.147.945.645.0
Judge acceptance14.836.744.447.348.5

Refinement versus independent resampling. To test what feedback-guided revision adds beyond candidate selection, we compared NSPI with best-of-5 sampling on a stratified subset of 60 OSWorld tasks with Opus 5. Best-of-5 generates five candidate programs independently at temperature 1.0. Both methods use the same task-completion judge for selection. NSPI achieves 80.0% benchmark Pass^3, compared with 53.3% for best-of-5, a gain of 26.7 percentage points.

MetricBase agentBest-of-5NSPI
Pass^3 (%)73.353.380.0

Can you discuss theoretical aspects of the policy refinement method? Are there formal guarantees under reasonable assumptions? (Q1)

It is true that the contraction-mapping guarantee of MDP policy iteration does not apply here, but we can state an analogous guarantee under a no-regression assumption. If a proposed program never fails on a case the previous program succeeded on, the programs form a pre-order by the set of cases they solve, and that pre-order supplies a notion of monotonicity. The guarantee then becomes a question of how much of the search space must be traversed to find one improvement (i.e., a proposal that flips from failure to success). Modern LLMs are well suited to keeping this traversal small, since they propose targeted edits from the judge's diagnosis. Alternatives such as MCTS also apply, but they would likely need more proposals and are more complex to implement, so we only mention them here. In NSPI, no regression is weakly enforced by giving the healer the previous code and prompting it to preserve working behavior; a stricter check would require some notion of equivalence on the rollouts.

If the task is well scoped (e.g., "fill in this form with the information from this file") and the number of steps is bounded, the search space is finite (as opposed to “find the cases where this program does not terminate”). Roughly, if m cases fail initially and each proposal has probability at least q of fixing one more case without breaking others, the expected number of iterations is at most m/q. This holds for any improvement operator, whether or not it uses an LLM. We will state the assumptions and prove this bound in the Appendix.

AuthorsOpenReview ↗

Response to Reviewer wKNr comments (Part 3)

Why use a test-time iterative algorithm instead of training open-weight models? (Q2)

We discuss parameter-learning approaches at two stages: cross-task training before deployment and workflow-specific fine-tuning at test time. Our target setting requires adapting to a newly introduced workflow without a workflow-specific training set or ground-truth reward, while achieving high reliability across repeated executions (high Pass^k) at low amortized cost (Eqs. 1 and 3). These requirements motivate learning a reusable program from the agent’s first execution and refining it through a bounded number of judged runs. NSPI converts the initial trajectory into a program and uses subsequent execution feedback to refine it before reuse.

Before deployment. Cross-task training can strengthen an open-weight agent and give NSPI a better seed trajectory, but it cannot incorporate execution experience from a workflow that has yet to arrive. Our results also show that even frontier agents struggle to succeed consistently across repeated executions, motivating further adaptation. On OSWorld, Terra achieves 55.9% Pass^1 and 34.6% Pass^3 where all three runs must succeed (Table 1). Test-time refinement targets this reliability gap directly by using experience from the incoming workflow to improve the program that will execute it on later runs.

At test time. Fine-tuning an agent for each new workflow is one possible approach, but it adds a training cost. The resulting agent still incurs planning costs if it selects actions step by step. Collecting training rollouts is itself a bottleneck for online RL in GUI environments [7]. In contrast, NSPI uses the first trajectory and a bounded number of judged runs to build a program that can repeat the workflow without replanning every step. Our base-agent results provide a reference for the cost of repeated planning and the savings enabled by program reuse. For instance, on OSWorld with Terra, a single base-agent run costs \0.48,about67timesthe0.48, about 67 times the \\0.0072 cost of running the learned program. The program also improves Pass^3 from 34.6% to 50.4%, and its \$3.09 construction cost breaks even after 6.5 runs (Tables 1 and 3). Workflow-specific fine-tuning must amortize the costs of collecting training rollouts and updating model weights, while retaining recurring planning costs if the agent selects actions step by step. NSPI avoids weight updates and amortizes its bounded construction cost over inexpensive program executions.


References

[1] Chaudhuri et al. Neurosymbolic Programming, Foundations and Trends in Programming Languages. 2021.

[2] Mitchell. Machine Learning. 1997.

[3] Brown et al. Language Models are Few-Shot Learners. NeurIPS 2020.

[4] Sutton and Barto. Reinforcement Learning: An Introduction. 2018.

[5] Yang et al. Large Language Models as Optimizers. ICLR 2024.

[6] Brooks et al. Large Language Models can Implement Policy Iteration. NeurIPS 2023.

[7] Li et al. Fully Decoupled Reinforcement Learning with Adaptive Data Curation for GUI Agents. EMNLP 2026.

Reviewer_CSWmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper addresses the problem of unreliable and expensive execution of recurring computer-use tasks, where existing agents re-plan every step for each execution instead of reusing previous experience. It introduces Neuro-Symbolic Policy Iteration (NSPI), a framework that converts an agent's execution trajectory into a reusable program, preserves stable decisions such as action ordering and control flow in code, and delegates state-dependent decisions to neural models. The framework iteratively improves the policy by executing it, diagnosing failures using task-completion and step-level judges, and revising the code using additional agent exploration. The paper evaluates NSPI on OSWorld and ScienceBoard, reporting improvements in reliability across repeated executions, better transfer to parameter and initial-state variations, and substantial reductions in per-run cost and latency compared with existing baselines.

Critical strengths
  1. The paper addresses an important problem with computer-use agents, where agents repeatedly plan each step of a workflow from scratch even when the same procedure needs to be executed multiple times with only minor changes in inputs or starting states, which makes execution both expensive and unreliable. The idea of converting execution experience into a reusable policy is intuitive, and the proposed neuro-symbolic representation provides a way to preserve stable decisions in code while leaving state-dependent decisions to neural models.
  2. The proposed NSPI framework provides a systematic approach to improving these policies by executing them, diagnosing failures, and revising the code using feedback from judges and additional agent exploration, rather than relying only on the initial demonstration. The paper also evaluates the approach across OSWorld and ScienceBoard, considers both repeated-execution reliability and generalization to different task instances, and reports substantial reductions in inference cost and latency, making the approach practically relevant for recurring workflows.
Critical weaknesses

1.The central idea of representing recurring workflows as reusable programs is intuitive, but it remains unclear how much of the contribution comes from the proposed policy iteration framework as opposed to existing ideas in programmatic automation, iterative code repair, and state-dependent execution, particularly given that the closest baseline, AutoRPA, also learns reusable scripts from agent interactions. Although the paper compares against several baselines, a clearer separation of the benefits of iterative refinement, neural primitives, and pre-action verification would help establish which components are essential for the reported improvements. 2.The approach relies heavily on LLM-based judges to determine whether a policy succeeds and to identify the first point of failure, which is then used to guide policy refinement, but the reported judge analysis shows that these judges reject many reliable policies and struggle to localize failures that are not visible on the screen. This raises the question of how sensitive the iterative refinement procedure is to incorrect or incomplete feedback, particularly when failures involve hidden application state or incorrect intermediate actions, and whether the method can reliably improve policies when its own feedback mechanism is imperfect. 3. The generalization results are promising, but the evaluation primarily considers variations of the same workflow through changes in task parameters and initial states, where the policy is refined on generated variants before being tested on held-out original tasks. It is therefore unclear how well the approach generalizes to more substantial changes in workflow structure, previously unseen application states, or tasks that require a different sequence of operations, especially since the results on longer workflows show that converting lengthy agent trajectories into code can introduce errors and reduce task success compared with the base agent.

Questions for authors

Q1, Since AutoRPA also learns reusable scripts from agent interactions, what is the main methodological contribution that distinguishes NSPI from existing approaches, and can the authors isolate how much of the reported improvement comes from iterative refinement rather than the programmatic policy representation or the neural primitives used at execution time?

Q2. Given that the policy refinement loop depends on LLM-based judges that can reject reliable policies and fail to localize errors that are not visible on the screen, how robust is NSPI to incorrect judge feedback, and can the authors demonstrate that the iterative refinement process still converges to reliable policies when the judges provide incomplete or incorrect diagnoses?

Q3. The paper demonstrates transfer across parameter and initial-state variations but reports reduced performance on longer workflows, so what properties of a task determine whether converting an agent trajectory into a reusable policy is beneficial, and can the authors establish when the gains in execution reliability and cost outweigh the loss of flexibility and the additional errors introduced during policy construction?

3 comments
AuthorsOpenReview ↗

Response to Reviewer CSWm comments (Part 1)

Thank you for your time and constructive suggestions! We have addressed all your comments below.


How does NSPI differ from AutoRPA, and what does refinement add? (W1/Q1)

NSPI is a general way of learning a reliable policy from one request. NSPI is designed as a general method for learning reliable reusable policies from individual user requests. AutoRPA focuses on efficient automation within a known task family, using multiple instances and their benchmark evaluators to construct and repair a reusable script. NSPI instead asks how an agent can turn experience from an incoming request into a policy that executes it reliably, using only the request and interaction with the environment. This motivates its single-request initialization, potentially unsuccessful seed trajectory, and judge-guided refinement without ground-truth reward feedback.

We further compared both methods in this single-task setting using Terra on a stratified sample of 81 OSWorld tasks, with the same seed trajectory for each method. Final-policy Pass^3 was 48.1% for NSPI versus 25.9% for AutoRPA, demonstrating the generality of NSPI.

Contribution of framework components. The initial neuro-symbolic policy combines code with neural primitives for locating elements and checking conditions on the live screen. To isolate the contribution of refinement, we evaluate the following configurations on a fixed set of 60 OSWorld tasks with Opus 5, keeping the policy representation and available primitives unchanged. Judge-guided refinement raises Pass^3 by 23.3 percentage points, and adding agent continuation provides a further 5.0-point gain.

ConfigurationPass^3 (%)
Base agent73.3
Initial neuro-symbolic policy50.0
+ judge-guided refinement73.3
+ agent continuation during refinement78.3

The pre-action verifier is an optional deployment check and was disabled for all main results, including the ablation above. We evaluate it separately in Section 4.6 and will clarify its optional role and use in the experimental settings in the manuscript.


How robust is refinement to incorrect judge feedback? (W2/Q2)

How often does a judge error cost a working policy? On OSWorld with Terra, the judges accepted only 2.5% of incorrect policies, showing that the judges act as a strict gate against invalid policies. On the other hand, if the judges reject a correct policy, further unnecessary code healing could break a working policy. However, we show that NSPI is robust to this noise: among OSWorld tasks where the judges rejected such an intermediate policy, the final policy still passed all three runs on 128/204 tasks (62.7%) with Terra and 214/229 (93.4%) with Opus 5. Thus, refinement can still produce reliable final policies despite false rejections.

Does refinement still improve reliability despite judge errors? Despite judge errors, Pass^3 improved in all four settings from iteration 1 to iteration 5, with gains of 8.3–25.2 percentage points. For this analysis, we used best-so-far policy selection based on task-completion judge feedback and measured the selected policy’s benchmark Pass^3 at each iteration. Pass^3 rose or held steady at every iteration in three settings and peaked at iteration 3 on ScienceBoard with Opus 5.

SettingMetric (%)Iter. 1Iter. 2Iter. 3Iter. 4Iter. 5
OSWorld · TerraPass^344.353.554.856.557.3
OSWorld · Opus 5Pass^347.465.168.769.072.6
ScienceBoard · TerraPass^320.120.120.721.928.4
ScienceBoard · Opus 5Pass^323.139.147.945.645.0

Can the step judge diagnose failures that are invisible on screen? Some actions return a value without visibly changing the screen. On the 16 annotated failed rollouts in Table 6, giving the same judge model access to program return values doubled exact failure localization from 5/16 to 10/16. With return values, the judge identified the failing step in 5/6 cases where the failure was invisible on screen. We will update Section 4.7 to include this comparison and clarify that the original judge did not have access to program return values.

AuthorsOpenReview ↗

Response to Reviewer CSWm comments (Part 2)

When is conversion worthwhile? (W3/Q3)

We clarify that the reviewer's concern about the challenges of long workflows and when NSPI is beneficial are two separate questions:

  1. Given a task, is it beneficial to apply NSPI to find a reusable policy for this task?
  2. How can we improve NSPI for long workflows?

We address both in the following.

Recurring workflows. Conversion is worthwhile when a procedure recurs often enough to repay construction and its inputs can be read afresh on each run. In general, conversion is worthwhile if the expected number of times the task repeats is larger than the break-even point stated in Section 2.1. For example, transferring records between applications follows a common sequence even as the records change. Process-mining work identifies such recurring interaction sequences and uses execution frequency to assess their suitability for automation [1]. At the average costs in Table 3, per-run savings recover the construction cost after 4–7 executions on OSWorld and 12–27 on ScienceBoard. Note that the cost benefit of a reusable policy does not necessarily depend on the length of the task. A long task that runs deterministic actions over hundreds of entries can be represented by a loop (at high cost efficiency), while a short task where every step requires an LLM call for state-dependent decision-making may not be worth converting to a code policy at all.

Reuse when the procedure changes. We show that code policies learned by NSPI can be reused for different but related tasks (e.g., changing operation sequence). On two groups of ScienceBoard tasks with Opus 5, we compared two ways to build the initial policy for a new request. One converted an agent trajectory collected on the target task into a code policy (i.e., cold start following NSPI). The other used two policies learned for related requests as a warm start. With this warm start, a single model call produced a policy that achieved Pass^3=1 on five of seven tasks in each application, without collecting a trajectory or refining the policy on the target task.

ApplicationTarget tasksReliable init policy with cold startReliable init policy with warm start
TeXstudio705
ChimeraX735

What accounts for the OSWorld-V2 drop? A recurring conversion error was failing to preserve dependencies on values observed earlier in a long trajectory. Across 106 tasks, 47 initial policies stopped because the program could not derive a required value. In 12 of 16 losses we examined, a later action depended on a value the agent had read from the screen but the policy could not recover.

Refinement recovered reliable execution on 13 tasks whose initial policies failed, with each final policy succeeding in all three evaluations. However, the full Pass^1 remained below the base agent’s (17.45% vs. 28.04%, Table 4). One potential way to preserve these dependencies would be to convert the trajectory one subtask at a time and make each subtask return the observed values that later steps need.


References

[1] Leno et al. Robotic Process Mining: Vision and Challenges. Business & Information Systems Engineering, 2021

Thank you for addressing my questions and concerns, I plan to stick to my current score!

Comments on the paper

1 comment

We thank all reviewers for their thoughtful reviews and constructive feedback. We summarize the discussion below.

Our contribution is reliable, efficient recurring computer use through reusable neuro-symbolic policies. Across four settings, NSPI achieves the highest Pass^3 among compared methods, improving over the base agent by 3.6–15.8 percentage points at 15–217× lower per-run cost after construction.

IssueKey takeaway and supporting evidence
Contribution vs. AutoRPA (CSWm)Reliable policies can be constructed from individual requests without benchmark feedback. NSPI outperforms AutoRPA from shared seed trajectories (48.1% vs. 25.9% Pass^3).
Component contributions (CSWm, 7epF)Both refinement components improve reliability: initial policy → judge-guided refinement → agent continuation yields 50.0% → 73.3% → 78.3% Pass^3.
Judge robustness (CSWm)Refinement improves independently measured reliability despite imperfect judges: iteration-5 Pass^3 exceeds iteration 1 by 8.3–25.2 points across all four settings.
Failure diagnosis (CSWm)Program return values make failures easier to diagnose: supplying them to the same judge doubles exact failure localization (5/16 → 10/16).
Generalization (CSWm, 7epF)Policies transfer from variants to held-out originals, exceeding AutoRPA by 20 points on both parameter and initial-state variations when judges receive return values.
Cross-task reuse (CSWm)Learned policies accelerate adaptation to related requests: across two applications, warm starts increase reliable initial policies from 3/14 to 10/14, using one policy-generation call per request without target-task exploration or refinement.
Execution budgets (7epF)ScienceBoard budgets are aligned across units: base-agent planning-call limits map to 5×/16× state-changing primitive-action caps, applied during both refinement and final evaluation.
Verifier scope (7epF)Pre-action verification is optional and disabled for all main results, including reported costs. Its separate evaluation in §4.6 and optional role will be made explicit in the manuscript.
Learning and objective (wKNr)Expected Pass^k (Eq. 1) is the reliability objective for the learned program. NSPI revises it using judge verdicts and diagnoses as surrogate feedback, without access to benchmark-evaluator verdicts during refinement.
PI framing (wKNr)Core claims stand independently of classical PI. We will rename the method Neuro-Symbolic Program Iteration and revise §3.1.1 to distinguish conceptual inspiration from classical PI and its guarantees.
Theoretical aspects (wKNr)Outlined a conditional m/q expected-iteration argument under explicit no-regression and improvement-probability assumptions, without claiming classical PI guarantees.
Open-weight training (wKNr)Program reuse complements stronger trained agents: code preserves stable decisions that stepwise agents repeatedly infer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.