acceptodds
Under review as a conference paper at ICLR 2027

Synchronizing Work with Results in Long-Horizon Computer-Use Agents via Demand-Linked Execution

Abstract

Long-horizon computer-use agents (CUAs) carry out work across applications, using findings and intermediate conclusions to make decisions and update artifacts. These results must remain connected to the work that needs them and to the earlier decisions that rely on them as execution moves on. Yet work can remain unfinished after a needed result arrives, and earlier decisions can go unreviewed when that result changes. We identify this phenomenon as Result–Work Desynchronization (RWD) and analyze its associated decision costs. In this paper, we propose Demand-Linked Execution (DLE) to mitigate the RWD phenomenon by preserving result uses together with each work’s evolving state. DLE records why results are requested and which decisions and artifacts rely on each adopted version. When results arrive or change, the agent’s model uses these records and current software observations to resume waiting work and determine which earlier outcomes to retain or revise. On representative long-horizon CUA benchmarks spanning desktop (OSWorld 2.0), web (Odysseys), and mobile (MobileWorld) environments, DLE improves task success over Vanilla by 6.5, 9.0, and 10.9 percentage points, respectively. Controlled probes compare DLE with baselines that make the current result’s use explicit or maintain a persistent task graph. DLE’s revision advantage persists as intervening work increases.

Primary area:
applications to robotics, autonomy, planning
Keywords:
LLM Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nJSROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the desynchronization between outcomes and ongoing tasks in long-horizon computer-use agents—specifically, instances where tasks fail to resume after an outcome is returned, or where dependent tasks are not re-evaluated following the revision of a previous outcome. DLE (Dependency-aware Lifecycle Management) mitigates this by tracking task states, preserving the context of returned outcomes, and recording versioned dependency relationships; this enables the model to resume tasks and re-examine affected decisions and objects. The approach demonstrates performance improvements across three benchmarks (desktop, web, and mobile) and two model configurations, with its efficacy further validated through latency probes and ablation studies.

Critical strengths
  1. The problem holds practical value. The paper distinguishes between "returning to a previous state" and "revising after adoption," accurately characterizing two types of work resumption—often overlooked in long-horizon tasks—that require attention.
  2. The methodology is well-structured. By incorporating versioning, usage context, and current work status, the approach facilitates effective review, allowing users to retain valid prior decisions and avoid redundant rework.
  3. The validation is comprehensive. The experiments encompass three interface environments and two model configurations, and include paired-task analysis, delayed-probe testing, and component ablation studies.
Critical weaknesses
  1. RWD events and decision costs were annotated by GPT-5.6, without human review or consistency assessment. It is recommended that a stratified sample of these trajectories undergo independent human review, with documentation provided to support this process.
  2. The distinction between this approach and versioned dependency graphs, truth maintenance systems, or existing state management solutions remains insufficiently clear—could you please elaborate? Additionally, does the current implementation rely on nested calls, logical switching, or actual concurrent execution?
  3. The lifecycle of the queue—specifically regarding successive revisions and reviews—is not sufficiently explained. Please clarify how previous reviews are consolidated or cleared when transitioning through versions (e.g., v1→v2→v3), how reviews that conclude out of sequence are handled, and what system guarantees are in place for scenarios involving missed detections or the misidentification of an entity’s origin. 4.Although the paper compares mechanism-level baselines—such as memory, replanning, and dependency management—it does not yet provide a direct comparison with closely related frameworks like MACU and StructAgent.
Questions for authors

Refer weakness

2 comments
AuthorsOpenReview ↗

Response to Reviewer nJSR (1/2)

Thank you for your detailed review. We address your concerns below.

W1. RWD events and decision costs were annotated by GPT-5.6, without human review or consistency assessment. It is recommended that a stratified sample of these trajectories undergo independent human review, with documentation provided to support this process.

We checked both the episode labels in Table 1 and the decision-cost attribution in Figure 2.

Two of the authors independently annotated 200 availability episodes and all 91 post-use revision episodes from the OSWorld 2.0 trajectories generated by Qwen3.8-27B. Availability episodes were randomly sampled within the original GPT label strata, selecting 60 RWD and 140 non-RWD episodes. Both authors applied the criteria in Appendix B.2, then resolved disagreements from the trajectory evidence before the GPT labels were revealed.

For Figure 2, they also annotated every outer decision in 32 complete runs, with eight runs randomly selected from each Vanilla/Local Intervention x successful/unsuccessful group.

Table R1. Human review of the annotations. All rates are percentages. Human agreement is between the two authors before resolving disagreements. P/R denotes GPT-5.6 precision/recall against the finalized human labels.

Annotation targetAudited unitsHuman agreementGPT-5.6 P/R
Availability episodes200 episodes94.593.3/91.8
Post-use revision episodes91 episodes93.492.6/89.3
Decision attribution3,768 decisions96.090.7/93.1

We then recomputed the episode rates and decision costs using the human labels.

Table R2. Measurements before and after human review. Availability is reweighted to the original GPT RWD/non-RWD proportions. Revision uses all 91 original episodes. Decision costs use identical audited runs and pooled decision denominators in both columns.

QuantityGPT-5.6 (%)Human (%)
Availability: RWD labels, reweighted to original strata28.929.5
Post-use revision: RWD labels, all 91 episodes29.730.8
Decision cost: Vanilla, unsuccessful runs28.727.1
Decision cost: Vanilla, successful runs15.916.6
Decision cost: Local Intervention, unsuccessful runs24.923.8
Decision cost: Local Intervention, successful runs13.213.7

Human review identifies 28 revision episodes with RWD, compared with 27 originally. RWD-related work still accounts for roughly one quarter of decisions in unsuccessful Vanilla runs and one sixth in successful runs.

AuthorsOpenReview ↗

Response to Reviewer nJSR (2/2)

Weakness 2.

Weakness 4.

DLE records what each result version is used for and restores that use with the current work state when the result changes. This connects the update to the judgment or artifact that needs attention. For example, a changed course time can require a calendar edit while the semester assignment remains valid. These execution rules build on dependency tracking, truth maintenance, and provenance (Section 3).

The implementation uses one active work at a time. A call saves its intended use and continuation before switching to the callee. A return brings the answer and saved use back to the caller's latest state. Nested requests therefore use logical switching within the same model (Algorithm 1 and Appendix A.1).

We compared full MACU and StructAgent on all 108 OSWorld 2.0 tasks using Qwen3.8-27B for every model role, including MACU's manager and workers. MACU allows up to four parallel workers. Task starts, scoring, inference allocation, and the shared 500-call budget are matched across methods. Each framework retains its native interfaces.

We also tested an enhanced Versioned Graph in DLE's runtime. It retains completed nodes and historical versions, supports dynamic dependencies and transitive invalidation, and permits fine-grained nodes and selective retention. Reviews receive the affected node's state, dependency explanations, and history. DLE restores the individual adopted uses directly.

Table R4. OSWorld comparison. Each method is evaluated once on all 108 tasks. Costs average all attempts, including failures, and cover every model role and retry. Tokens count inputs and outputs. Time covers preparation through delivery. VM-minutes sum occupancy across VMs.

MethodBinary (%)Model callsTokens (M)Time (min)VM-minutes
MACU22.2386.02.7727.278.7
StructAgent24.1329.42.0941.041.0
Versioned Graph25.0303.91.9936.736.7
DLE25.9259.91.1731.631.6

DLE completes 28 tasks versus 24 for MACU, using 32.7% fewer calls and 57.8% fewer tokens. MACU finishes sooner through parallel execution, at the cost of greater cumulative VM occupancy.

We further compared revision handling on 80 episodes at delays of 0, 20, 50, and 100 decisions. Each method starts before adoption and forms its own records. Success requires all necessary changes before renewed reliance or delivery, while preserving two initially correct, unaffected targets per trial.

Table R5. Revision handling. Each method has 320 trials and 640 unaffected targets. Call counts cover the entire trial, including setup, intervening work, and review.

MethodCorrect revision (%)Unaffected targets preservedModel calls/trial
MACU66.6621/640 (97.0%)161.2
StructAgent63.8614/640 (95.9%)138.9
Versioned Graph70.3626/640 (97.8%)115.5
DLE78.8633/640 (98.9%)96.1

Comment:

Weakness 3. Please clarify how previous reviews are consolidated or cleared when transitioning through versions (e.g., v1→v2→v3), how reviews that conclude out of sequence are handled, and what system guarantees are in place for scenarios involving missed detections or the misidentification of an entity’s origin.

The queue groups affected uses by work, source, last assessed version, and new version (Appendix A.2). Repeated notifications add uses to the same entry. If v3 arrives while the v2 review is pending, it gets a separate entry, and the original adopted basis is preserved.

Closing v2 leaves the v3 review pending. If v3 is reviewed first, an older entry whose uses are already covered can be closed without moving their reviewed version back or repeating edits. Reviews awaiting further evidence stay open.

Once the model identifies and registers a source update, the runtime queues affected recorded uses, including declared transitive dependencies (§5.3). Missed updates or missing links can leave work unreviewed, and incorrect source matches can trigger unnecessary reviews. Those reviews do not automatically require an edit.

Reviewer_64r4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies Result-Work Desynchronization: agents may fail to resume work when a needed result arrives or fail to revisit earlier decisions when that result changes. It proposes Demand-Linked Execution, which maintains:

  • Work State describing current judgments, evidence, and unresolved questions.
  • Result Return records connecting prerequisite requests to their intended uses.
  • Version-specific Use Links that identify work affected by later corrections. DLE is evaluated on OSWorld 2.0, Odysseys, and MobileWorld using two Qwen models, alongside controlled delay probes and component ablations.
Critical strengths
  • RWD is a concrete, well-motivated problem. The paper isolates a specific failure (results that arrive or change without the dependent work being resumed or revisited), measures how common it is in real trajectories, and builds a method aimed directly at it.
  • The evaluation is broad and reasonably controlled. It covers complete task sets across three environments and two model sizes. The controlled delay probes also test the proposed failure mode more directly than aggregate benchmark scores alone.
Critical weaknesses
  • Lack of comparison against other task-graph based approaches. Dynamic task-graph systems such as MACU can propagate new findings and revise downstream work. Although the paper includes an author-designed Explicit Dependency baseline, it does not compare against MACU or a task graph augmented with versioned dependencies and invalidation. Therefore, it remains unclear whether DLE provides an advantage beyond a sufficiently expressive task-graph implementation.
  • The existing ablations show that DLE’s components are useful, but do not isolate whether the gains specifically result from preventing RWD. The paper does not separately test the completion guard, version tracking, or a compute-matched structured-state baseline, nor does it measure whether DLE actually reduces validated RWD incidence.
Questions for authors
  • How well do GPT-5.6 RWD annotations agree with expert humans and other model family? Need to justify the choice of this judge.
  • Can you report multiple runs per method, with variance and paired tests against the strongest baseline on each benchmark?
  • Do the gains hold with a stronger backbone, such as a frontier proprietary model?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 64r4 (1/3)

Thank you for your detailed review. We address your concerns below.

W1. Although the paper includes an author-designed Explicit Dependency baseline, it does not compare against MACU or a task graph augmented with versioned dependencies and invalidation. Therefore, it remains unclear whether DLE provides an advantage beyond a sufficiently expressive task-graph implementation.

W2. The paper does not separately test the completion guard, version tracking, or a compute-matched structured-state baseline […].

We compared DLE with full MACU and StructAgent and an enhanced Versioned Graph. The graph retains structured node states, completed nodes, historical versions, dynamic dependencies, and transitive invalidation, with selective retention in DLE's shared runtime.

All methods use Qwen3.8-27B with matched task starts, scoring, inference allocation, and a shared 500-call limit. The full frameworks retain their native interfaces, and MACU uses the same model for its manager and parallel workers. All roles count toward the budget. Revision trials start before adoption, so each method builds its own records.

Table R4. Task performance, revision handling, and inference cost. Binary and per-task costs use one evaluation of all 108 tasks. Revision results and calls/trial use 320 trials from 80 episodes at four delays, with 640 initially correct, unaffected targets. Costs include all roles and failed attempts.

MethodBinary (%)Calls/taskTokens (M)/taskCorrect revision (%)Unaffected targets preservedCalls/trial
MACU22.2386.02.7766.6621/640 (97.0%)161.2
StructAgent24.1329.42.0963.8614/640 (95.9%)138.9
Versioned Graph25.0303.91.9970.3626/640 (97.8%)115.5
DLE25.9259.91.1778.8633/640 (98.9%)96.1

DLE has the highest task and revision success in these comparisons, with fewer model calls and tokens. Compared with Versioned Graph, it completes 27 more revision trials while using fewer calls per trial. This complements the affected-work-lookup ablation in Section 6.4, which retains version and use records while removing automatic consumer enumeration and pending-review presentation.

Comment:

W2. The existing ablations show that DLE’s components are useful, but do not isolate whether the gains specifically result from preventing RWD. [...] Nor does it measure whether DLE actually reduces validated RWD incidence.

Using the annotation and human-review procedure described in Q1, we measured RWD in the original OSWorld trajectories underlying the paper’s Tables 2 and 3.

Table R6. RWD incidence and task performance. Episode entries show RWD/eligible counts and rates. Decision shares pool all runs per method. Partial scores are unchanged from the paper.

MethodAvailability RWDRevision RWDRWD + rework decisions (%)OSWorld Partial (%)
Feedback-based Correction506/2,387 (21.2%)35/104 (33.7%)18.352.3
DLE w/o automatic affected-work lookup270/2,451 (11.0%)37/116 (31.9%)14.152.9
DLE249/2,476 (10.1%)18/119 (15.1%)8.957.6

DLE has fewer RWD episodes for both transition types despite reaching more eligible transitions. At the task level, 31 of the 42 tasks with higher Partial scores than Feedback-based Correction also have fewer RWD episodes.

In the lookup ablation discussed in W1, revision RWD rises from 15.1% to 31.9% while Partial falls from 57.6% to 52.9%. Availability RWD remains near 10–11%, so the increase is concentrated in post-use revision.

AuthorsOpenReview ↗

Response to Reviewer 64r4 (2/3)

Q1. How well do GPT-5.6 RWD annotations agree with expert humans and other model family? Need to justify the choice of this judge.

We compared GPT-5.6 with independent human annotations and Qwen3.8-Max-0902, using the same trajectory material and annotation criteria.

Two of the authors independently annotated 200 availability episodes and all 91 post-use revision episodes from the OSWorld 2.0 trajectories generated by Qwen3.8-27B. The availability sample was drawn at random within the original GPT label strata, with 60 RWD and 140 non-RWD episodes. Both authors used Appendix B.2 and resolved disagreements from the trajectory evidence before seeing the GPT labels. They also annotated all 3,768 outer decisions in 32 complete runs, with eight runs sampled from each Vanilla/Local Intervention x successful/unsuccessful group. Qwen annotated the same material without access to either GPT or human labels.

Table R1. Agreement with humans and across model families. All rates are percentages. Human agreement is measured before resolving disagreements. P/R denotes precision/recall against the finalized human labels. The last column directly compares GPT-5.6 and Qwen3.8-Max.

Annotation targetAudited unitsHuman agreementGPT-5.6 P/RQwen3.8-Max P/RGPT–Qwen agreement
Availability episodes200 episodes94.593.3/91.883.3/90.291.0
Post-use revision episodes91 episodes93.492.6/89.375.0/75.087.9
Decision attribution3,768 decisions96.090.7/93.180.9/81.092.6

GPT-5.6 is closer to the human annotations on both precision and recall, supporting our choice of judge. We also recomputed the episode rates and decision costs under each set of labels to examine whether the judge changes the observations in Section 4.

Table R2. Measurements using each set of labels. Availability is reweighted to the original GPT RWD/non-RWD proportions. Revision uses all 91 original episodes. Decision costs use identical audited runs and pooled decision denominators across the three columns.

QuantityGPT-5.6 (%)Qwen3.8-Max (%)Human (%)
Availability: RWD labels, reweighted to original strata28.932.129.5
Post-use revision: RWD labels, all 91 episodes29.730.830.8
Decision cost: Vanilla, unsuccessful runs28.726.327.1
Decision cost: Vanilla, successful runs15.916.616.6
Decision cost: Local Intervention, unsuccessful runs24.923.923.8
Decision cost: Local Intervention, successful runs13.215.213.7

The two models differ on individual labels, while both recover the main pattern: RWD is frequent, and its associated work remains substantial even in successful runs. The human reference supports the same observations.

AuthorsOpenReview ↗

Response to Reviewer 64r4 (3/3)

Q2. Can you report multiple runs per method, with variance and paired tests against the strongest baseline on each benchmark?

We ran five additional evaluations on the full task sets (108 OSWorld, 200 Odysseys, and 201 MobileWorld tasks), comparing DLE with the strongest baseline for each metric in the original Table 2. All methods used Qwen3.8-27B, with decoding, budgets, and evaluation criteria unchanged.

Table R3. Five new runs per method with Qwen3.8-27B. Scores (%) are mean ± SD across runs. The 95% BCa CIs describe task-level uncertainty for these five runs. Bootstrap samples retain all five repetitions of each sampled task. P-values are from two-sided paired permutation tests, with Holm correction across the five comparisons.

Benchmark / metricComparatorDLE, mean ± SDComparator, mean ± SDGain [95% CI], ppAdjusted p
OSWorld / BinaryExplicit Dependency26.3 ± 3.224.8 ± 6.2+1.5 [-3.1, +6.1]1.000
OSWorld / PartialFeedback-based Correction60.2 ± 1.656.3 ± 0.8+3.9 [+0.9, +7.0]0.046
Odysseys / PerfectExplicit Dependency45.9 ± 3.045.0 ± 3.2+0.9 [-2.4, +4.2]1.000
Odysseys / Rubric AverageFeedback-based Correction70.6 ± 1.467.4 ± 1.1+3.2 [+0.7, +5.6]0.046
MobileWorld / Overall SRExplicit Dependency29.3 ± 3.323.8 ± 3.7+5.5 [+2.7, +8.6]0.002

The gains in OSWorld Partial and MobileWorld success are positive in all five runs, and the Odysseys Rubric Average gain is positive in four. All three gains over the original Table 2 baselines remain significant after Holm correction. The smaller Binary and Perfect differences have intervals spanning zero.

Comment:

Q3. Do the gains hold with a stronger backbone, such as a frontier proprietary model?

DLE’s gains also hold with GPT-6 Sol at medium reasoning effort, compared with the strongest original baseline for each metric.

Table R7. Results with a stronger backbone. Scores (%) average five runs.

MethodOSWorld BinaryOSWorld PartialOdysseys PerfectOdysseys Rubric Avg.MobileWorld SR
Strongest baseline per metric40.672.068.090.453.5
DLE45.277.674.392.261.6

The baseline is Explicit Dependency for Binary, Perfect, and MobileWorld SR, and Feedback-based Correction for Partial and Rubric.

DLE improves strict completion over the strongest baseline by 4.6, 6.3, and 8.1 percentage points on OSWorld, Odysseys, and MobileWorld, respectively. Partial and Rubric scores also improve.

Reviewer_npiSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies Result–Work Desynchronization (RWD) in long-horizon computer-use agents: waiting work is not resumed when a needed result arrives, and earlier decisions/artifacts are not reviewed when an adopted result is corrected.

Critical strengths

Problem framing matches a real CUA failure mode, with concrete incidence and cost. Table 1 reports RWD in 28.9% of eligible episodes and 79.6% of analyzed Vanilla runs on OSWorld 2.0 (Qwen3.8-27B). Figure 2 attributes roughly one quarter / one sixth of decisions in unsuccessful / successful Vanilla runs to RWD-related rework.

Critical weaknesses

W1. Novelty overstated vs MACU, StructAgent, and classical TMS/provenance; Explicit Dependency understates MACU. RWD risks rebranding known long-horizon failure modes.

W2. RWD labels (Table 1 / Figure 2) are GPT-5.6-only with no human agreement, yet form Contribution 1.

W3. Main baselines are mechanism adaptations of MemGPT / AdaPlanner / CRITIC / LLMCompiler, not full MACU / StructAgent / Agent S2; absolute scores remain low.

W4. No variance, multi-seed, or significance on main tables; OSWorld Binary +2.8 pp vs Explicit Dependency is ~3 tasks on 108.

W5. Source-linking left to the model with no error analysis; delay probes are scripted; Odysseys live-web reproducibility undiscussed; no token/time metrics; post-use revision is rare observationally (91 episodes) while the narrative leans on it.

Questions

  1. Human agreement / precision–recall for GPT-5.6 RWD labels on ≥100 stratified episodes?
  2. Same-backbone comparison to full MACU and StructAgent?
  3. Source-link error rate and effect of controlled link noise on Figure 5?
  4. Bootstrap CIs / McNemar for Table 2 and task-level comparisons?
  5. Fraction of control/review/return vs native GUI decisions; do reviews explain the 13 MobileWorld regressions?
Questions for authors

see above

3 comments
AuthorsOpenReview ↗

Response to Reviewer npiS (1/3)

Thank you for your detailed review. We address your concerns below.

W1. Novelty overstated vs MACU, StructAgent, and classical TMS/provenance; Explicit Dependency understates MACU.

Q2. Same-backbone comparison to full MACU and StructAgent?

We compared full MACU and StructAgent on all 108 OSWorld 2.0 tasks using Qwen3.8-27B for every model role, including MACU's manager and workers. MACU allows up to four parallel workers. Task starts, scoring, inference allocation, and the shared 500-call budget are matched across methods. Each framework retains its native interfaces.

DLE records what each result version is used for and restores that use with the current work state when the result changes. This connects the update to the judgment or artifact that needs attention. For example, a changed course time can require a calendar edit while the semester assignment remains valid. These execution rules build on dependency tracking, truth maintenance, and provenance (Section 3).

The original Explicit Dependency baseline is a mechanism-level adaptation following LLMCompiler (Section 6.1). We also tested an enhanced Versioned Graph in DLE's runtime. It retains completed nodes and historical versions, supports dynamic dependencies and transitive invalidation, and permits fine-grained nodes and selective retention. Reviews receive the affected node's state, dependency explanations, and history. DLE restores the individual adopted uses directly.

Table R4. OSWorld comparison. Each method is evaluated once on all 108 tasks. Costs average all attempts, including failures, and cover every model role and retry. Tokens count inputs and outputs. Time covers preparation through delivery. VM-minutes sum occupancy across VMs.

MethodBinary (%)Model callsTokens (M)Time (min)VM-minutes
MACU22.2386.02.7727.278.7
StructAgent24.1329.42.0941.041.0
Versioned Graph25.0303.91.9936.736.7
DLE25.9259.91.1731.631.6

DLE completes 28 tasks versus 24 for MACU, using 32.7% fewer calls and 57.8% fewer tokens. MACU finishes sooner through parallel execution, at the cost of greater cumulative VM occupancy.

We further compared revision handling on 80 episodes at delays of 0, 20, 50, and 100 decisions. Each method starts before adoption and forms its own records. Success requires all necessary changes before renewed reliance or delivery, while preserving two initially correct, unaffected targets per trial.

Table R5. Revision handling. Each method has 320 trials and 640 unaffected targets. Call counts cover the entire trial, including setup, intervening work, and review.

MethodCorrect revision (%)Unaffected targets preservedModel calls/trial
MACU66.6621/640 (97.0%)161.2
StructAgent63.8614/640 (95.9%)138.9
Versioned Graph70.3626/640 (97.8%)115.5
DLE78.8633/640 (98.9%)96.1

AuthorsOpenReview ↗

Response to Reviewer npiS (2/3)

W2 / Q1. RWD labels are GPT-5.6-only with no human agreement.

Human agreement / precision–recall for GPT-5.6 RWD labels on ≥100 stratified episodes?

The human review covers 291 episodes: 200 availability episodes and all 91 post-use revision episodes. Two of the authors annotated them independently under the criteria in Appendix B.2. Availability episodes were randomly sampled within the original GPT label strata, with 60 RWD and 140 non-RWD episodes. The authors resolved disagreements from the trajectory evidence before the GPT labels were revealed.

To check Figure 2 as well, the same authors independently annotated all 3,768 outer decisions in 32 complete runs, with eight randomly selected runs from each Vanilla/Local Intervention x successful/unsuccessful group. All reviewed trajectories were generated by Qwen3.8-27B on OSWorld 2.0.

Table R1. Human agreement and GPT-5.6 precision/recall. All rates are percentages. Human agreement is between the two authors before resolving disagreements. P/R is measured against their finalized labels.

Annotation targetAudited unitsHuman agreementGPT-5.6 P/R
Availability episodes200 episodes94.593.3/91.8
Post-use revision episodes91 episodes93.492.6/89.3
Decision attribution3,768 decisions96.090.7/93.1

We then recomputed the episode rates and decision costs using the human labels.

Table R2. Effect of human relabeling. Availability is reweighted to the original GPT RWD/non-RWD proportions. Revision uses all 91 original episodes. Decision costs use identical audited runs and pooled decision denominators in both columns.

QuantityGPT-5.6 (%)Human (%)
Availability: RWD labels, reweighted to original strata28.929.5
Post-use revision: RWD labels, all 91 episodes29.730.8
Decision cost: Vanilla, unsuccessful runs28.727.1
Decision cost: Vanilla, successful runs15.916.6
Decision cost: Local Intervention, unsuccessful runs24.923.8
Decision cost: Local Intervention, successful runs13.213.7

On the complete revision set, the human labels identify 28 RWD episodes, compared with 27 in the original annotations. RWD-related work still accounts for roughly one quarter of decisions in unsuccessful Vanilla runs and one sixth in successful runs.

Comment:

W4 / Q4. No variance, multi-seed, or significance on main tables; OSWorld Binary +2.8 pp vs Explicit Dependency is ~3 tasks on 108.

Bootstrap CIs / McNemar for Table 2 and task-level comparisons?

We ran five additional evaluations on the full task sets (108 OSWorld, 200 Odysseys, and 201 MobileWorld tasks), comparing DLE with the strongest baseline for each metric in the original Table 2. All methods used Qwen3.8-27B, with decoding, budgets, and evaluation criteria unchanged.

Table R3. Five new runs per method with Qwen3.8-27B. Scores (%) are mean ± SD across runs. The 95% BCa CIs describe task-level uncertainty for these five runs. Bootstrap samples retain all five repetitions of each sampled task. P-values are from two-sided paired permutation tests, with Holm correction across the five comparisons.

Benchmark / metricComparatorDLE, mean ± SDComparator, mean ± SDGain [95% CI], ppAdjusted p
OSWorld / BinaryExplicit Dependency26.3 ± 3.224.8 ± 6.2+1.5 [-3.1, +6.1]1.000
OSWorld / PartialFeedback-based Correction60.2 ± 1.656.3 ± 0.8+3.9 [+0.9, +7.0]0.046
Odysseys / PerfectExplicit Dependency45.9 ± 3.045.0 ± 3.2+0.9 [-2.4, +4.2]1.000
Odysseys / Rubric AverageFeedback-based Correction70.6 ± 1.467.4 ± 1.1+3.2 [+0.7, +5.6]0.046
MobileWorld / Overall SRExplicit Dependency29.3 ± 3.323.8 ± 3.7+5.5 [+2.7, +8.6]0.002

The gains in OSWorld Partial, Odysseys Rubric Average, and MobileWorld success over the original Table 2 baselines remain significant after Holm correction. The smaller OSWorld Binary and Odysseys Perfect differences have intervals spanning zero.

For the original OSWorld Binary result of 28 versus 25 successes, the exact two-sided p-value is at least 0.25. For the original MobileWorld comparison, the 25 DLE-only and 13 baseline-only successes give p = 0.073.

AuthorsOpenReview ↗

Response to Reviewer npiS (3/3)

W5. Source-linking left to the model with no error analysis; delay probes are scripted; […] post-use revision is rare observationally (91 episodes) while the narrative leans on it.

Q3. Source-link error rate and effect of controlled link noise on Figure 5?

We manually checked 180 source-correspondence decisions from 36 DLE trajectories on OSWorld 2.0. Seven (3.9%) were incorrect: four revisions were registered as new sources, and three distinct sources were merged.

We tested their effect by overriding one correction's source assignment in 4, 8, or 16 of the 80 revision episodes, using the same selected episodes at each delay. A missed match registers a new source. A wrong match assigns the correction to another existing source. Initial states, correction content, and budgets are held fixed.

Table R8. Revision success under source-assignment errors. Rates use all 320 trials. The clean condition is the matched revision evaluation in R5.

Corrupted episodesMissed source match (%)Wrong source (%)
0/8078.878.8
4/80 (5%)76.675.0
8/80 (10%)72.571.6
16/80 (20%)65.963.4

At 10% episode corruption, DLE retains 71.6–72.5% revision success, above the unperturbed Versioned Graph result of 70.3% in R5.

The original 91 revision episodes span 31 of 108 Vanilla runs (Table 1). The probes control when a correction arrives and how much independent work intervenes. Agents start before adoption, construct their own records, and receive the correction through an ordinary observation or tool return. Scoring checks the resulting work, with correct source mappings and consumer lists withheld from the agents (Appendix B.3).

Comment:

W5. Odysseys live-web reproducibility undiscussed.

For Odysseys, we grade the saved screenshots and action sequences, so later changes to live pages do not affect reproducing the reported scores.

Comment:

Q5. Fraction of control/review/return vs native GUI decisions; do reviews explain the 13 MobileWorld regressions?

We count each outer decision once by its dispatched operation. State or use declarations emitted alongside a native action do not add another decision.

Table R9. DLE decision shares across all attempted tasks. Native includes interface checks and repairs performed during review. Control covers calls, resumption, and completion decisions. Request retries contribute to the model-call totals in R4, but are not additional outer decisions.

BenchmarkDecisionsNative (%)Control (%)Review (%)Return (%)
OSWorld27,67888.12.78.11.1
Odysseys15,48683.85.67.13.5
MobileWorld7,84388.64.94.22.3

Six of the 13 MobileWorld regressions involved source matching or review: two wrong source associations, one unnecessary edit, and three cases of budget exhaustion during review. The other seven were native interaction failures (five) or planning errors (two).

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overall Response to Reviewers

We thank the reviewers for their careful reading and thoughtful comments. We have responded in all three reviews and welcome any further questions or discussion before the discussion closes.

A central concern raised by the reviewers was how DLE differs from explicit versioned-dependency graphs. In long-horizon, cross-application tasks, dependencies between results and decisions are many-to-many: one result may inform multiple decisions, while each decision may rely on multiple results. In versioned-dependency graphs, tracking which specific uses a revision affects requires fine-grained nodes and additional dependency bookkeeping. DLE instead directly links each result version to its actual uses and the corresponding work state, enabling selective revision: pending work resumes when results arrive, and only affected uses are revisited when results change. As reported in our responses, DLE achieves 78.8% vs. 70.3% revision success, with 96.1 vs. 115.5 model calls per trial, compared with the enhanced versioned-graph baseline.

AuthorsOpenReview ↗

Revised Manuscript

As the discussion period comes to a close, we’ve uploaded the revised manuscript with the new analyses and results we reported in our responses to all three reviewers. The main additions are marked in dark blue, including the annotation checks in Section 4 and the new experiments and comparisons in Section 6.

Thank you all for your thoughtful questions and suggestions. Your feedback helped us describe the RWD phenomenon more clearly, explain how DLE mitigates it, and strengthen the evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.