acceptodds
Desk rejected at ICLR 2027

WorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolution

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Scaling interactive environments has become increasingly important for training long-horizon agents through reinforcement learning. These environments often require further testing and refinement after initial generation, and correcting defects in a task may also require updating its work context and verifier. However, existing construction pipelines offer limited support for such iterative refinement while keeping these components aligned. In this paper, we introduce WorkRover, a framework for training long-horizon cowork agents through recursive environment evolution. Its harness-driven synthesis enables construction agents to jointly build and validate semantically aligned work contexts, executable tasks, and evidence-grounded verifiers while they adapt their actions to workspace state and execution feedback rather than follow a fixed construction workflow. To support environment evolution, we introduce Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions through dependency-closed replay and selects the smallest validated repair among tested candidates while preserving task requirements and evaluation constraints. To provide finer-grained training signals, we propose Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with progress signals derived from deterministic workspace checks and assigns a bounded share of positive-progress credit to prerequisite turns using recorded tool-output and artifact dependencies. Using WorkRover, we post-train Qwen3.5-35B-A3B to obtain WorkRover-35B-A3B and evaluate it on several cowork-agent benchmarks. Its reported scores exceed those of the base model by 26.27 percentage points on JobBench, 29.32 percentage points on OfficeQA Pro, and 413.25 Elo points on GDPval v2. These results suggest that WorkRover offers a promising foundation for scalable training of long-horizon cowork agents.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Cowork agents, Long-horizon agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vyhFOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WORKROVER, a training framework for long-horizon cowork agents, consisting of three components: (1) a harness-driven environment synthesis pipeline that constructs work context, long-horizon tasks, and multi-layer verifiers; (2) Factorial Compatibility Replay (FCR), which enumerates candidate repair combinations and selects the one that changes the fewest components while passing replay and safety gates; (3) Verifier-to-Turn Credit Assignment (VTCA), which uses a frozen deterministic checker to measure per-turn workspace state progress/regression, and propagates positive progress credit to genuinely causal prior turns along tool-output version dependency chains. The authors conduct post-training (SFT+RL) on Qwen3.5-35B-A3B, and evaluate on five external benchmarks, claiming comprehensive leadership among 27–35B scale models.

Critical strengths

This paper proposes the FCR mechanism that provides a concrete, actionable, and thoroughly validated solution to the real but previously overlooked problem of coupled environment component repair. Across both natural-defect and artificially-injected-defect cohorts, FCR significantly outperforms intuitive baselines such as Full-regeneration, Local-only, and Apply-all (SRR 70.3% vs 60.7%, intent drift only 1.3% vs 14.3%, cross-task regression reduced from 7.0% to 0.7%). Additionally, the appendix's blind dual-annotator plus third-party-adjudicated credit-localization audit and the enforced separation of sealed audit shards and calibration shards, provides internal evidence for the effectiveness of the VTCA mechanism.

Critical weaknesses
  1. The authors need to clarify how the different baseline scores were evaluated and conduct a fairer comparison. Existing baseline scores come from different evaluation protocols (different harness versions, different task subsets, and mixed metrics with different semantics), with some data directly extracted from vendor model card self-reported values. The authors should clarify the reproduction details of the different evaluation protocols.

  2. The authors' experimental analysis should be strengthened by comparing with the fair evaluation protocol. The currently repeatedly emphasized core figures are based on protocols that are not directly comparable, rather than controlled retesting under uniform conditions, weakening the persuasiveness of the core claim of "WorkRover leading across the entire 27-35B group."

  3. All experiments are validated on only a single backbone model (Qwen3.5-35B-A3B), within a single model family, with no cross-model-family validation whatsoever.

Questions for authors
  1. In tables such as JobBench, AutomationBench, and OfficeQA Pro, various baseline models use different harness versions, judge models, or even different metric definitions. Could at least the same-scale (27-35B) open-source baselines be re-evaluated under a unified harness and judge configuration?

  2. Could at least one other model reproduction result be provided to demonstrate the effectiveness of the FCR/VTCA methods themselves, rather than overfitting to a specific model family's training data distribution?

  3. FCR has higher token cost relative to Apply-all/Local-only (1.00 vs 0.59/0.43); in a real large-scale environment evolution scenario involving tens of thousands of task packages, is this doubled cost still practical?

3 comments

Response to Reviewer vyhF

We thank you for your constructive feedback and will revise the paper to improve clarity and exposition.

vyhF-W1. Baseline provenance and reproduction details.

Table 1 mixes two kinds of entries. Method-level comparisons (backbone / SFT / RL) are our own runs with the Harness, split, judge, scoring rule, and inference limits fixed within each suite. External references are quoted from the original public reports — official leaderboards, model cards, and technical reports — with their settings disclosed as in Appendix C.1. We deliberately do not re-run public baselines: an independent reproduction would add our own Harness and judge variation on top of theirs and make the comparison less, not more, comparable. What we add is an explicit source column (who produced the score, release/snapshot, metric, disclosed configuration).

SuiteSource of reference scoresConfiguration that must accompany the score
JobBenchOfficial leaderboard, model reportsLeaderboard Main uses 65 tasks, OpenCode v1.14.18, Grok 4.3. Qwen3.6-27B's 21.8 is a model-card report with split/Harness/judge unspecified.
GDPvalArtificial Analysis GDPval v2, Sep 12 snapshotAA references use v2/Stirrup. Occamy's 1128 uses a different 207-task cohort and judge panel; Nex-N2-mini's 1097.03 is a supplied record.
AutomationBenchOfficial Public results, model reports, AAPublic strict pass (600 tasks) differs from AA objective completion (657 private tasks).
WorkBuddy OfficeBenchmark paper, leaderboard50-task, three-run Claude Code setting.
OfficeQA ProOfficial repository, vendor reports, our runsPro split, document access, Harness, numerical tolerance, attempts reported separately.

vyhF-W2. Controlled evidence and the scope of the leadership claim.

Our method claims rest on matched controls, not on the backbone-to-final-model gap. The appendix is organized as a chain of questions, each with the conditions held fixed and the result observed:

QuestionControlResult
Do construction components help?Table 31, six component removals + three assembly strategies, same frozen agentRemoving validation-guided rewriting: yield 74.8% → 49.8%; removing evolution: → 63.7%
Rewriting vs. filtering only?Table 32, same intent IDs, seeds, capsAdmission 49.8% → 74.8%; success over all intents 22.2% → 43.9%
Does repair recover the same candidate stream?Table 29, same candidates and budget capsAdmission: no repair 49.8% / full regeneration 68.8% / FCR 74.8%
Compatibility selection with identical proposals?Tables 2, 30, shared-proposal panelSafe resolution 64.0% → 73.0%, regression 7.0% → 0.7% vs. Apply-all
Full repair procedure?Table 30, end-to-end, each method generates its own repairsvs. full regeneration: 60.7% → 70.3%, drift 14.3% → 1.3%, relative tokens 1.34 → 1.00
Are gains from relaxed safeguards?Table 15, analysis-only removal of each safeguardRemoving regression gate: raw 73.0% → 78.0% but safe 70.3% → 65.0%
Do gains persist over revisions?Table 16, Fig. 7dThree generations: 70.3% / 72.4% / 71.8%; five FCR iterations: 35→65%
Learning gain beyond shared SFT?Fig. 6 / Table in 41tB-Q2, identical SFT init+3.39 pp, +43 Elo, +5.67 pp, +2.63 pp, +5.37 pp
Which credit component?Table 22Local progress +1.8 pp; provenance +2.0 pp over local-only, +2.2 pp over random recipients
Is it just extra compute?Tables 22, 24, computed-matched54.0% → 59.0% at 1.08× compute; 54.0% → 58.2% at matched compute
Does the signal survive independent checking?Tables 23, 25, sealed audit + independent checks + blinded judgePrecision/recall 0.94/0.78; agreement 96.0%; 57.8% vs. official 59.0%

These controls support the method claims independently of published rankings.

vyhF-W3. Additional 8B/14B results and model-family coverage.

We add Qwen3-8B and Qwen3-14B, each reported at backbone, SFT-only, SFT + episode-only GRPO, and SFT + VTCA under the same per-suite evaluation configuration. The two RL variants share the SFT initialization.

BackboneStageJobBench (%)GDPval v2 (Elo)AutomationBench (%)WorkBuddy (%)OfficeQA Pro (%)
Qwen3-8BBackbone5.6684.33.143.410.8
Qwen3-8BSFT only14.3939.68.451.520.5
Qwen3-8BSFT + E-GRPO20.11054.613.055.627.8
Qwen3-8BSFT + VTCA22.61097.616.758.932.3
Qwen3-14BBackbone8.2754.84.653.515.4
Qwen3-14BSFT only20.01010.012.161.427.9
Qwen3-14BSFT + E-GRPO27.41125.018.365.336.6
Qwen3-14BSFT + VTCA30.51168.023.268.241.8
Qwen3.5-35B-A3BSFT + E-GRPO33.50117523.0071.5043.50
Qwen3.5-35B-A3BSFT + VTCA36.89121828.6774.1348.87

VTCA improves all five endpoints at all three scales. All three checkpoints belong to the Qwen3.5 family, so we describe this as cross-scale validation.

vyhF-Q1. Can the same-scale baselines be compared under one Harness and judge?

For all experiments we run, yes: each suite follows its official evaluation implementation with the agent Harness, split, evaluator/judge, scoring rule, and inference limits fixed within the comparison — the convention for LLM evaluation, and the only way the scores stay comparable to the published numbers they are placed beside. For external 27–35B references we use the original public reports rather than our own re-runs (W1).

vyhF-Q2. Reproduction beyond this backbone.

On the internal endpoint with FCR-refined environments fixed, VTCA raises pass rate from 37.0% → 41.0% (+4.0 pp) at 8B and 46.8% → 51.1% (+4.3 pp) at 14B (35B: 54.0% → 59.0%, Table 22). The environment × credit ablation results below compare training with and without iterative FCR under both credit methods.

Training environmentRL credit8B internal pass (%)14B internal pass (%)
Initial validated pool, no iterative FCREpisode-only GRPO34.043.8
FCR-refined poolEpisode-only GRPO37.046.8
Initial validated pool, no iterative FCRVTCA38.048.1
FCR-refined poolVTCA41.051.1

As public context for RL at this scale, NVIDIA's Nemotron-Cascade mixture trains on SWE-rebench; its reported SWE-bench Verified change (Agentless, pass@1 over four generations) is +5.6 pp for 8B and +3.5 pp for 14B. These use Qwen3-derived models and a different method, so direct evidence for VTCA remains our matched experiments.

vyhF-Q3. Repair cost relative to Apply-all and large-scale practicality.

Under shared proposals the costs are FCR = 1.00 and Apply-all = 1.25: FCR uses 20% fewer model tokens than Apply-all while resolving more defects and regressing less.

TrackMethodSafe resolution (%)Regression (%)Relative tokens
Shared proposalsLocal-only42.03.20.43
Shared proposalsGreedy-sequential58.02.70.78
Shared proposalsApply-all64.07.01.25
Shared proposalsFCR73.00.71.00
End-to-endFull regeneration60.78.21.34
End-to-endFCR70.31.11.00

The extra cost is against Local-only (2.33×), which resolves 31 pp fewer defects; that is a quality–cost tradeoff, not a scalability problem. At the scale the reviewer raises, total cost is driven by how much work is done per candidate, and FCR is cheaper than both Apply-all and full regeneration. Enumeration is bounded: the Harness screens incompatible combinations before materialization, Verifier alternatives reuse execution evidence when the closed Context–Task–Harness state and seed are identical, and each invocation has at most 16 direct combinations.

Thank you for your response. After careful consideration, I have decided to keep the postive score.

Reviewer_41tBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WorkRover, a framework for training long-horizon co-work agents through recursive environment evolution. WorkRover generally consists of two components: Factorial Compatibility Replay (FCR), which uses rollout feedback to evaluate combinations of component revisions through dependency-closed replay and selects the smallest validated repair among tested candidates while preserving task requirements and evaluation constraints, and Verifier-to-Turn Credit Assignment (VTCA), which augments trajectory-level GRPO advantages with signals from deterministic workspace-progress checks and assigns a bounded share of positive-progress credit to prerequisite turns using recorded tool-output and artifact dependencies. Empirical results on various agentic co-work benchmarks with Qwen3.5-35B-A3B demonstrate the effectiveness of the proposed method.

Critical strengths

The authors have spent much effort designing the measurement protocol. Regarding the repair-selection question, they conducted experiments with proposals frozen across methods, where FCR's joint selection raises natural SRR from 64.0% to 73.0% over Apply-all while cutting cross-task regression from 7.0% to 0.7% (Table 2). The safeguard ablations show the information-boundary gate is what prevents leakage-driven raw resolves (82.0% raw resolution drops to 59.0% SRR without it; Table 15).

Critical weaknesses
  1. Neither component is compared empirically against any published method.
  • On the repair side, the baselines (Local-only, Greedy-sequential, Apply-all, Full-regeneration; Tables 2, 29–30) are all author-constructed. Some dicussions with recent works like FACET [1] or Envs-FORGE [2] should be welcome.
  • On the learning side, the sole baseline is episode-only GRPO rather than any existing turn-level credit method (e.g., TRACE [3]; TRCA [4]; SPA-RL [5]).

This matters because the paper's own controls already hint the proposed signal may not be the differentiator: most of the VTCA gain comes from adding local progress at all (B 54.0 → C 55.8, Table 22), with provenance-based propagation contributing only about 2 pp over random recipients (E 57.8 vs. D 55.6).

  1. It is hard to attribute the improvement of the proposed method to task quality or distiilation. The reported gains (+26.27pp for JobBench, and +413.25 Elo for GDPval v2) are computed over the raw Qwen3.5-35B-A3B backbone (Table 1) while including an SFT stage distilled from 30,115 Qwen3.5-397B-A17B trajectories. The method-attributable deltas from the matched study are +3.4 pp and +43 Elo for VTCA over episode-only GRPO (Fig. 6), which is an order of magnitude smaller.

References

[1] FACET: Preserving source intent and executable state in terminal task synthesis. arXiv preprint arXiv:2608.18580.

[2] Envs-FORGE: Frontier-optimized reward-grounded environment synthesis for agent RL. arXiv preprint arXiv:2608.14312, 2026.

[3] TRACE: Turn-level reward assignment via credit estimation for long-horizon agents. arXiv preprint arXiv:2607.13988, 2026.

[4] TRCA: Transition-wise rubric credit assignment for long-horizon LLM agents. arXiv preprint arXiv:2608.16156, 2026.

[5] SPA-RL: Reinforcing LLM agents via stepwise progress attribution. arXiv preprint arXiv:2505.20732, 2025.

Questions for authors
  1. (Clarification) Could you run a same-budget comparison of VTCA against one published turn-level credit method (e.g., TRACE-style or GiGPO-style) on the internal endpoint? Given that E–D is only ~2 pp (Table 22), this is the comparison that would establish the provenance mechanism — and equivalently, an adaptation of one published repair procedure (FACET or Envs-FORGE) as an FCR baseline.
  2. (Clarification) Please state the SFT-only (no RL) scores on the five external suites in a table; Figure 6's bars cannot be read precisely, and this would quantify the teacher-distillation share of the headline gains.
  3. (Clarification) Please report per-seed dispersion and clustered intervals for the five external-suite deltas in Figure 6 (VTCA vs. episode-only GRPO, +2.6 to +5.7 pp): do these hold across the five training seeds under the C.5 protocol?
  4. (Clarification) Given the 0.77–1.07 pp margins over Occamy-1.0 on OfficeQA Pro and AutomationBench, could you re-evaluate both systems under a matched protocol (same GDPval task cohort and judge panel, same AutomationBench scoring mode)? As it stands the cross-model comparison mixes cohorts, judges, and scoring conventions that the paper itself documents in C.1.
  5. (Clarification) Is there any overlap analysis between the synthesized training tasks (built from deep-research sessions and a 36,064-job-skill inventory) and the five external benchmarks, particularly JobBench and GDPval deliverables? Contamination could inflate the transfer gains.
  6. (Suggestion) Reporting FCR's repair quality as a function of checker coverage (supported weight 0.65) and dependency recall (0.78; Table 23) would help bound applicability to settings with sparser provenance.
6 comments

Response to Reviewer 41tB

We thank you for your constructive feedback and will revise the paper to improve clarity and exposition.

Critical Weaknesses

41tB-W1. Published-method comparisons and the distinct cowork problem.

(a) Positioning and repair baselines. Table 1 evaluates the resulting system against public proprietary and open-weight models; Tables 2 and 29–30 isolate repair decisions with controlled alternatives. These answer different questions.

The relevant distinction is the problem being tested. Cowork tasks combine heterogeneous deliverables, mutable workspace and application state, and acceptance evidence distributed across related artifacts. A reference execution can establish feasibility without establishing acceptance of all legitimate alternatives, consistency across source versions, or validity of other tasks after a shared component changes. Terminal methods also address executable-state consistency; their existing mechanisms should be adapted to these requirements, rather than judged by directly comparing unrelated benchmark scores. Our distinction is the explicit maintenance and credit-allocation mechanism, not the absence of prior cowork environments.

(b) Comparison with published repair and learning mechanisms. The empirical adaptations are specified in Q1.

MethodPublished mechanismRelevant comparison with WorkRover
FACETGrounds terminal task artifacts in shared executable state and routes failures to targeted repair.FCR compares frozen revision combinations, includes dependency-induced updates, and checks affected tasks before selecting a minimal validated repair.
Envs-FORGESelects synthesis actions near a learning frontier and jointly rewrites task bundles.FCR preserves the original outcome requirements while restoring compatibility; a constrained adaptation must disclose this objective change.
TRACEUses frozen-model reference-answer likelihood changes for turn credit.VTCA checks observable workspace states; a TRACE adaptation needs a declared answer representation for structured deliverables.
TRCACombines deterministic transition rubrics with cumulative coverage increments.VTCA combines signed current-state progress with recipient selection through artifact/output use. Deterministic feedback itself is not unique to VTCA.
SPA-RLLearns a progress estimator for reward redistribution.Its adaptation must charge estimator training and inference; VTCA uses frozen executable progress checks.
GiGPOGroups actions from repeated anchor states for step-level relative advantages.A cowork adaptation must specify state equivalence across mutable artifacts and report anchor-state coverage.

The common-initialization and common-input method adaptations in Q1 separately test credit assignment and repair.

41tB-W1. Published-method comparisons and the distinct cowork problem (continued).

(c) What Table 22 actually establishes. The full improvement over the turn-balanced terminal baseline is 59.0 − 54.0 = 5.0 pp. Adding local progress accounts for 1.8 pp along the reported sequence, or 36% of that difference; the table does not support the statement that most of the full gain comes from local progress alone.

ContrastInternal pass rates (%)Difference (pp)Question tested
C − B55.8 − 54.0+1.8Adding local progress.
E − C57.8 − 55.8+2.0Adding provenance propagation without trace cost.
E − D57.8 − 55.6+2.2Selecting provenance recipients instead of random recipients.
D − C55.6 − 55.8−0.2Redistributing progress to random earlier turns.
F − E59.0 − 57.8+1.2Adding trace cost.
F − no propagation, cost retained59.0 − 57.0+2.0Propagation with trace cost present.

D and E share the producing-turn mass, propagation budget, recipient count, and eligible-event mask; only the earlier recipient identities differ. Random redistribution does not improve on local-only progress in these aggregates, while provenance selection adds 2.2 pp over that control. These are conditional mechanism contrasts: E−D must not be added again to C−B, E−C, and F−E. They support recipient selection within our configuration, but do not establish superiority to published credit methods or significance without paired uncertainty.

41tB-W2. Task quality, teacher distillation, and the scale of the attributed gain.

The headline gains describe the complete training pipeline. The SFT stage uses 30,115 demonstrations generated by Qwen3.5-397B-A17B; it is therefore essential to expose the intermediate checkpoint. The reported scores give the following stage-by-stage accounting:

ContrastJobBench (pp)GDPval v2 (Elo)Interpretation
SFT only − backbone+14.38+255.25Observed SFT-stage gain, including teacher behavior and selected task data.
SFT + episode-only GRPO − SFT only+8.50+115Episode-level RL after the same SFT stage.
SFT + VTCA − SFT + episode-only GRPO+3.39+43Incremental VTCA comparison with initialization and rollout budget matched.
SFT + VTCA − backbone+26.27+413.25Complete pipeline gain.

The SFT increment does not isolate the contribution of teacher distillation, because teacher behavior, task content, and demonstration selection change together. Q2 already gives all five training conditions numerically in this response.

We also report evaluation of the exact demonstration teacher under the same external-suite protocols:

TeacherJobBench (%)GDPval v2 (Elo)AutomationBench (%)WorkBuddy Office (%)OfficeQA Pro (%)
Qwen3.5-397B-A17B27.090513.068.039.0

The public Qwen3.5-397B-A17B weights identify a candidate checkpoint; the evaluation must match the revision that generated our demonstrations. Teacher scores contextualize the student–teacher gap, but cannot by themselves separate environment quality from distillation. That requires holding the teacher, initial intent stream, demonstration-generation budget, filtering rules, student initialization, and training setup fixed while varying the environment-construction/refinement procedure.

Questions for Authors

41tB-Q1. Same-budget comparisons with published credit and repair methods.

We include two focused adaptations of published methods: a TRACE-style turn-level credit baseline on the internal endpoint, and FACET-style targeted repair as an FCR baseline. These comparisons use common cowork inputs so that differences in task distributions do not determine the outcome.

Turn-level credit. We compare episode-only GRPO, a TRACE-style adaptation, and VTCA from the same SFT checkpoint, using the same frozen training pool, paired training seeds, terminal reward, optimizer settings, tuning allowance, and held-out internal evaluator. The adaptation retains TRACE's frozen-reference log-ratio state values and temporal-difference turn credit. For structured deliverables, the reference representation and its handling of equivalent valid outputs are fixed on development tasks. We report both matched attempted-rollout budgets and a separate matched-total-compute comparison; reference-model scoring, progress checks, and state capture are included in resource accounting.

Credit methodRollout-matched internal pass (%)Realized compute, relative to episode-onlyTotal-compute-matched internal pass (%)
Episode-only GRPO, turn-balanced54.01.0054.0
TRACE-style adaptation57.41.0856.6
VTCA, turn-balanced59.01.0858.2

The episode-only and VTCA rows reproduce the reference aggregates in Tables 22/24; the TRACE row reports the adapted baseline under the same evaluation conditions. Table 22's E−D contrast is 57.8 − 55.6 = 2.2 pp: it holds propagated credit mass, recipient count, and eligible events fixed while changing recipient identities. This isolates the contribution of provenance-based recipient selection within VTCA. The published-method comparison assesses complete credit methods; it complements this mechanism control rather than replacing it.

Environment repair. We adapt FACET's targeted-repair procedure, retaining failure routing to the implicated component and revalidation from a clean state. FACET-style repair and FCR receive the same initial defect cohort, original task requirements, proposer model, diagnostic evidence, and proposal/replay budget caps. Each method generates its own repairs, and a common independent final audit assesses resolution, intent preservation, and affected-task regressions. Failed attempts and retries remain in the cost ledger.

Repair methodNatural-defect safe resolution (%)Intent drift (%)Affected-task regression (%)Relative model-token cost
Full regeneration60.714.38.21.34
FACET-style targeted repair62.72.72.40.96
FCR70.31.31.11.00

The full-regeneration and FCR rows are the end-to-end reference results from Table 30; the FACET row reports the targeted-repair adaptation. This adaptation tests a published repair procedure directly; Local-only and Greedy-sequential remain separate design controls.

41tB-Q2. SFT-only scores and the numerical values in Figure 6.

Figure 6 already prints the numerical scores, but the similar colors make the training conditions difficult to distinguish. For immediate readability, we reproduce all five conditions as a table below.

BenchmarkBackboneSFT only (no RL)Backbone + E-GRPOSFT + E-GRPOSFT + VTCAVTCA − E-GRPO after SFT
JobBench (%)10.6225.0019.0033.5036.89+3.39 pp
GDPval v2 (Elo)804.751060.00960.001175.001218.00+43 Elo
AutomationBench (%)6.0015.5011.5023.0028.67+5.67 pp
WorkBuddy (Office) (%)60.5068.0065.0071.5074.13+2.63 pp
OfficeQA Pro (%)19.5534.0028.0043.5048.87+5.37 pp

These are the values used in the current Figure 6. They separate the SFT-stage gain from subsequent RL: for example, SFT adds 14.38 pp on JobBench and 255.25 Elo on GDPval, while VTCA adds 3.39 pp and 43 Elo over episode-only RL from the same SFT initialization. The SFT increment includes teacher behavior, task content, and demonstration selection; it does not uniquely identify a teacher-distillation contribution.

Reviewer_ndFbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents WorkRover, which iteratively refines the cowork environment while ensuring the alignment of the contexts, tasks, and verifiers. This paper also proposal to add turn level progress credit during RL process, rather than using only the trajactory rewards. This paper finetunes Qwen3.5-35B-A3B with SFT and then RL. Results show that this specificed fine-tuned model outperform the base backbond Qwen3.5-35B-A3B model on five cowork benchmarks.

Critical strengths

This paper proposes to iteratively refine the environment for agentic training data synthesis in cowork scenarios. This problem matters because defects in synthesized environments can lead to wrong rewards.

Critical weaknesses
  • Although environment defects can lead to wrong rewards during post-training, it remains unclear how much this problem affects training when environments are built with state-of-the-art construction methods (which I think can already produce reasonably well-constructed environments). The answer largely determines how significant the contribution is.
  • It is unclear where this paper wants to stand in the literature. There is already a large amounts of published work on constructing agent environments and on assigning step-wise rewards. The paper does not clearly explain what is new relative to these works.
  • The experimental comparison is unfair to some extent. The main results of this paper compares its fine-tuned model with the original backbone. If this paper want to show the improvement brought by the proposed "Recursive Environment Evolution" (btw, I think this term is somewhat overstated), it should compare models fine-tuned with and without the environment refinement, under the same training setup.
Questions for authors

See weaknesses.

5 comments

Response to Reviewer ndFb

We thank you for your constructive feedback and will revise the paper to improve clarity and exposition.

ndFb-1. How consequential are environment defects with recent construction methods?

1. Defects can corrupt training feedback. False acceptance rewards invalid outputs, false rejection penalizes valid solutions, and missing dependencies make tasks infeasible. Hacker-Fixer finds 323 of 1,968 terminal-benchmark tasks hackable without achieving their intended goals, demonstrating that verifier failures remain a practical concern. Hacker-Fixer

2. Strong construction still relies on validation and repair. In FACET's ablation, 48% (168/350) of accepted tasks required targeted repair. Cowork requires consistency across files and application state, acceptance of valid alternative deliverables, and compatibility after component revisions. A successful reference execution alone does not establish these properties. Existing coding/terminal checks provide useful foundations, but their coverage of these requirements must be verified. FACET

3. WorkRover addresses coordinated repair in cowork environments. FCR replays combinations of fixed Context, Task, Harness, and Verifier proposals, including dependency-induced updates, and selects the smallest validated combination that preserves task requirements and passes affected-task regression checks. On the same initial candidate stream under matched repair budget caps, admission is 49.8% without repair, 68.8% with full regeneration, and 74.8% with FCR (Table 29). With identical proposals, FCR also improves natural-defect safe resolution from 64.0% to 73.0% over Apply-all (Table 2). These results quantify construction and repair benefits.

ndFb-2. Positioning and the specific methodological contribution.

1. Positioning in the literature. WorkRover targets reliable post-training of long-horizon cowork agents operating across files, tools, and applications. Section 2, Appendix A.4–A.6, and Table 4 discuss its relationship to environment construction, repair, and stepwise credit assignment, including FACET, Envs-FORGE, TRACE, and TRCA.

2. Specific contributions. The Introduction identifies three contributions: jointly grounding Work Contexts, Tasks, and Verifiers under a dependency-tracking Harness; FCR, which compares fixed repair combinations and selects the smallest validated revision among those tested, including dependency-induced updates; and VTCA, which measures workspace progress and setbacks with frozen checks and allocates bounded positive credit through recorded tool-output and artifact-version use. The distinctions lie in the repair-selection procedure and the evidence used for credit allocation.

3. Why WorkRover remains needed. As explained in ndFb-1, cowork requires consistent evidence across heterogeneous artifacts, acceptance of legitimate alternatives, and compatibility after workspace revisions. Learning must also connect later deliverable changes to earlier actions that supplied their inputs. These requirements motivate our specific repair and attribution mechanisms alongside existing construction and reward methods.

4. Practical value. These designs arose from problems encountered during large-model cowork post-training. The paper contributes methods, controlled analyses, and practical lessons on preserving task requirements during repair, checking effects on dependent tasks, and learning from verifiable workspace changes.

ndFb-3. Fair comparisons, iterative refinement, and training attribution.

1. Table 1 provides broad system comparisons. Besides the backbone, it includes three proprietary models, four open-weight models above 400B, and five other 27–35B models. These provide system-level context; isolating refinement requires a separate matched-training comparison.

Direct comparison with published coding/terminal scores would mix task distributions, training setups, Harnesses, and metrics. The relevant methodological differences are summarized below (Table 4; Appendix A.4):

MethodSetting and objectiveConstruction / repair decision
FACETExecutable terminal-task synthesisGround artifacts in shared state; route validation failures to targeted component repair.
Envs-FORGEEnvironment synthesis near the policy's learning frontierSelect a synthesis action; jointly rewrite and verify the task bundle.
WorkRover / FCRReliable long-horizon cowork environmentsCompare fixed repair combinations, including dependency updates; minimize validated changes while preserving requirements and checking affected tasks.

A fair method-level comparison remains possible through adaptation to common cowork inputs, initialization, budgets, and evaluation. Our controlled repair alternatives isolate design choices; they are not reproductions of these published methods.

2. Existing experiments control the relevant factors.

  • Candidate recovery (Table 29): same initial candidates and repair budget caps; admission is 49.8% without repair, 68.8% with regeneration, and 74.8% with FCR.
  • Repair selection (Tables 2/30): identical frozen proposals and matched replay conditions; natural-defect safe resolution is 64.0% for Apply-all and 73.0% for FCR.
  • Complete repair procedures (Table 30): same defect cohorts, requirements, and budget caps; natural-defect safe resolution is 60.7% for regeneration and 70.3% for FCR.
  • Fixed-intent construction (Table 32): same 10,000 intents and frozen agent; filtering plus rewriting raises end-to-end success from 22.2% to 43.9%.
  • VTCA versus episode-only GRPO (Figure 6): same SFT initialization, task/seed schedules, and rollout budget; VTCA improves all five suites, including +3.39 pp on JobBench and +43 Elo on GDPval.
  • VTCA mechanism and resource controls (Tables 22/24): fixed credit budget and recipient count, plus a separate total-compute-matched comparison.

Matched budget caps do not imply identical realized consumption. Construction, repair, and policy-learning outcomes are reported separately.

3. “Recursive” refers to iterative environment refinement. Each validated revision becomes the starting point for the next diagnose–repair–validate cycle. Figure 7d reports rollout success (%) after successive repair rounds:

Method012345
Local-only35.039.041.042.043.043.0
Greedy-sequential35.042.047.050.052.053.0
Full regeneration35.043.048.052.055.057.0
FCR35.047.054.059.063.065.0

FCR improves by 30 pp overall and by another 18 pp after its first repair round. Separately, confirmed defects per 100 trigger rollouts decrease from 18 to 13 to 10 across three generations in 55 tracked workspaces (Table 16). These support repeated refinement within the evaluated horizon; they are environment-repair outcomes, not scores from separately retrained policies.

ndFb-3. Fair comparisons, iterative refinement, and training attribution (continued).

4. Error and training diagnostics. The following summarizes the reported errors and recomputed rates:

ScopeMetricComparison / result
End-to-end maintenance, Table 30Natural defects not safely resolvedNo repair 97.0% → FCR 29.7%
End-to-end maintenance, Table 30Task-intent driftRegeneration 14.3% → FCR 1.3%
End-to-end maintenance, Table 30Cross-task regressionRegeneration 8.2% → FCR 1.1%
Verifier audit, Table 27False acceptance / rejection1.93% / 1.43%; current audit, not a repair contrast
Training diagnostics, Table 24Invalid rolloutsE-GRPO 3.5% → VTCA 3.0%
Training diagnostics, Table 24Low-reward-variance groupsE-GRPO 17.5% → VTCA 11.0%
Training diagnostics, Table 24Zero-trajectory-advantage groupsE-GRPO 14.8% → VTCA 9.0%

The unresolved rate is 100% minus safe resolution among confirmed natural defects. Audit rates are 58/3,000 negative cases and 43/3,000 positive cases, with 37 invalid/unresolved cases counted separately. These metrics retain their respective denominators and cannot be combined into a single training-error rate.

5. With-versus-without-refinement training. The matched-training ablation isolates FCR during RL: both arms start from the same SFT checkpoint and use the same initial candidate stream, base constructor and admission rules, VTCA-GRPO algorithm, paired seeds, attempted-rollout budget, and evaluation. Only FCR is toggled. Effective training yield and realized costs are reported separately.

Matched-training ablation. Δ = with FCR − without FCR; percentage-rate differences are in percentage points.

MetricWithout FCRWith FCRΔ
JobBench (%) ↑33.2036.89+3.69
GDPval v2 (Elo) ↑11701218+48
AutomationBench (%) ↑24.3028.67+4.37
WorkBuddy Bench (Office) (%) ↑72.2074.13+1.93
OfficeQA Pro (%) ↑45.1048.87+3.77

Reward errors require independent labels; each diagnostic retains its own denominator. This comparison isolates RL-stage refinement, while the SFT stage is shared.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.