- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the desynchronization between outcomes and ongoing tasks in long-horizon computer-use agents—specifically, instances where tasks fail to resume after an outcome is returned, or where dependent tasks are not re-evaluated following the revision of a previous outcome. DLE (Dependency-aware Lifecycle Management) mitigates this by tracking task states, preserving the context of returned outcomes, and recording versioned dependency relationships; this enables the model to resume tasks and re-examine affected decisions and objects. The approach demonstrates performance improvements across three benchmarks (desktop, web, and mobile) and two model configurations, with its efficacy further validated through latency probes and ablation studies.
Critical strengths
- The problem holds practical value. The paper distinguishes between "returning to a previous state" and "revising after adoption," accurately characterizing two types of work resumption—often overlooked in long-horizon tasks—that require attention.
- The methodology is well-structured. By incorporating versioning, usage context, and current work status, the approach facilitates effective review, allowing users to retain valid prior decisions and avoid redundant rework.
- The validation is comprehensive. The experiments encompass three interface environments and two model configurations, and include paired-task analysis, delayed-probe testing, and component ablation studies.
Critical weaknesses
- RWD events and decision costs were annotated by GPT-5.6, without human review or consistency assessment. It is recommended that a stratified sample of these trajectories undergo independent human review, with documentation provided to support this process.
- The distinction between this approach and versioned dependency graphs, truth maintenance systems, or existing state management solutions remains insufficiently clear—could you please elaborate? Additionally, does the current implementation rely on nested calls, logical switching, or actual concurrent execution?
- The lifecycle of the queue—specifically regarding successive revisions and reviews—is not sufficiently explained. Please clarify how previous reviews are consolidated or cleared when transitioning through versions (e.g., v1→v2→v3), how reviews that conclude out of sequence are handled, and what system guarantees are in place for scenarios involving missed detections or the misidentification of an entity’s origin. 4.Although the paper compares mechanism-level baselines—such as memory, replanning, and dependency management—it does not yet provide a direct comparison with closely related frameworks like MACU and StructAgent.
Questions for authors
Refer weakness
2 comments
Response to Reviewer nJSR (2/2)
Weakness 2.
Weakness 4.
DLE records what each result version is used for and restores that use with the current work state when the result changes. This connects the update to the judgment or artifact that needs attention. For example, a changed course time can require a calendar edit while the semester assignment remains valid. These execution rules build on dependency tracking, truth maintenance, and provenance (Section 3).
The implementation uses one active work at a time. A call saves its intended use and continuation before switching to the callee. A return brings the answer and saved use back to the caller's latest state. Nested requests therefore use logical switching within the same model (Algorithm 1 and Appendix A.1).
We compared full MACU and StructAgent on all 108 OSWorld 2.0 tasks using Qwen3.8-27B for every model role, including MACU's manager and workers. MACU allows up to four parallel workers. Task starts, scoring, inference allocation, and the shared 500-call budget are matched across methods. Each framework retains its native interfaces.
We also tested an enhanced Versioned Graph in DLE's runtime. It retains completed nodes and historical versions, supports dynamic dependencies and transitive invalidation, and permits fine-grained nodes and selective retention. Reviews receive the affected node's state, dependency explanations, and history. DLE restores the individual adopted uses directly.
Table R4. OSWorld comparison. Each method is evaluated once on all 108 tasks. Costs average all attempts, including failures, and cover every model role and retry. Tokens count inputs and outputs. Time covers preparation through delivery. VM-minutes sum occupancy across VMs.
| Method | Binary (%) | Model calls | Tokens (M) | Time (min) | VM-minutes |
|---|---|---|---|---|---|
| MACU | 22.2 | 386.0 | 2.77 | 27.2 | 78.7 |
| StructAgent | 24.1 | 329.4 | 2.09 | 41.0 | 41.0 |
| Versioned Graph | 25.0 | 303.9 | 1.99 | 36.7 | 36.7 |
| DLE | 25.9 | 259.9 | 1.17 | 31.6 | 31.6 |
DLE completes 28 tasks versus 24 for MACU, using 32.7% fewer calls and 57.8% fewer tokens. MACU finishes sooner through parallel execution, at the cost of greater cumulative VM occupancy.
We further compared revision handling on 80 episodes at delays of 0, 20, 50, and 100 decisions. Each method starts before adoption and forms its own records. Success requires all necessary changes before renewed reliance or delivery, while preserving two initially correct, unaffected targets per trial.
Table R5. Revision handling. Each method has 320 trials and 640 unaffected targets. Call counts cover the entire trial, including setup, intervening work, and review.
| Method | Correct revision (%) | Unaffected targets preserved | Model calls/trial |
|---|---|---|---|
| MACU | 66.6 | 621/640 (97.0%) | 161.2 |
| StructAgent | 63.8 | 614/640 (95.9%) | 138.9 |
| Versioned Graph | 70.3 | 626/640 (97.8%) | 115.5 |
| DLE | 78.8 | 633/640 (98.9%) | 96.1 |
Comment:
Weakness 3. Please clarify how previous reviews are consolidated or cleared when transitioning through versions (e.g., v1→v2→v3), how reviews that conclude out of sequence are handled, and what system guarantees are in place for scenarios involving missed detections or the misidentification of an entity’s origin.
The queue groups affected uses by work, source, last assessed version, and new version (Appendix A.2). Repeated notifications add uses to the same entry. If v3 arrives while the v2 review is pending, it gets a separate entry, and the original adopted basis is preserved.
Closing v2 leaves the v3 review pending. If v3 is reviewed first, an older entry whose uses are already covered can be closed without moving their reviewed version back or repeating edits. Reviews awaiting further evidence stay open.
Once the model identifies and registers a source update, the runtime queues affected recorded uses, including declared transitive dependencies (§5.3). Missed updates or missing links can leave work unreviewed, and incorrect source matches can trigger unnecessary reviews. Those reviews do not automatically require an edit.
Response to Reviewer nJSR (1/2)
Thank you for your detailed review. We address your concerns below.
We checked both the episode labels in Table 1 and the decision-cost attribution in Figure 2.
Two of the authors independently annotated 200 availability episodes and all 91 post-use revision episodes from the OSWorld 2.0 trajectories generated by Qwen3.8-27B. Availability episodes were randomly sampled within the original GPT label strata, selecting 60 RWD and 140 non-RWD episodes. Both authors applied the criteria in Appendix B.2, then resolved disagreements from the trajectory evidence before the GPT labels were revealed.
For Figure 2, they also annotated every outer decision in 32 complete runs, with eight runs randomly selected from each Vanilla/Local Intervention x successful/unsuccessful group.
Table R1. Human review of the annotations. All rates are percentages. Human agreement is between the two authors before resolving disagreements. P/R denotes GPT-5.6 precision/recall against the finalized human labels.
We then recomputed the episode rates and decision costs using the human labels.
Table R2. Measurements before and after human review. Availability is reweighted to the original GPT RWD/non-RWD proportions. Revision uses all 91 original episodes. Decision costs use identical audited runs and pooled decision denominators in both columns.
Human review identifies 28 revision episodes with RWD, compared with 27 originally. RWD-related work still accounts for roughly one quarter of decisions in unsuccessful Vanilla runs and one sixth in successful runs.