acceptodds
Under review as a conference paper at ICLR 2027

WorkSynth: Off-Policy Synthetic Data for Professional Artifact Agents

Abstract

Professional work benchmarks such as GDPval have begun to evaluate language agents not by isolated answers, but by their ability to produce complete, usable artifacts such as reports, spreadsheets, slide decks, analyses, and decision documents. This shift exposes a gap in current post-training practice. While recent reinforcement-learning methods have achieved strong gains on tasks with compact and verifiable outcomes, professional artifact generation is characterized by underspecified goals, heterogeneous formats, domain-specific conventions, and evaluation criteria that are only partially reducible to pass–fail rewards. We argue that part of this limitation reflects a distributional mismatch: general instruction-tuning data rarely teaches models the forms of situated work required to transform complex requests and supporting materials into polished deliverables. This paper studies whether such capabilities can be improved through off-policy post-training rather than expensive online agentic reinforcement learning. We construct synthetic professional work supervision that varies along three dimensions: task formulation, artifact structure, and revision-oriented feedback. Across GDPval-style evaluations, we examine whether models benefit more from additional final-output demonstrations, from critique-and-revision supervision, or from data targeting specific artifact conventions and professional genres. Our results show that synthetic work-oriented supervision improves instruction adherence, structural completeness, and deliverable quality, with the largest gains arising when synthetic data represents not only the desired final artifact but also the evaluative criteria that distinguish acceptable from incomplete work. These findings suggest that professional artifact agents may be limited less by a lack of generic reasoning ability than by insufficient exposure to the conventions, constraints, and revision practices of real-world knowledge work.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language agents, professional artifact generation, off-policy post-training, synthetic work supervision

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_BthnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies post-training for professional artifact agents: benchmarks like GDPval evaluate agents by complete, usable deliverables rather than isolated answers, yet general instruction corpora rarely demonstrate how a situated request and supporting materials become a finished work product. The authors argue this capability gap is partly a distributional mismatch rather than a reasoning deficit, and propose WorkSynth, a pipeline that synthesizes professional work supervision from situated sources, varying task formulation, artifact genre and structure, and revision-oriented feedback. A tool-using teacher produces reference artifacts and trajectories, which are preflighted, scored by discriminative rubrics, audited at task level, and frozen; the student is then trained with assistant-token behavior-cloning SFT, with no sandbox or judge inside the gradient-update loop. Evidence comes in two deliberately separated branches: a professional-work branch demonstrates construction feasibility and exposes evaluator failures, and a software-artifact branch provides the controlled training study—frozen-trajectory SFT on a 9B student raises SWE-bench Verified resolution from 1.6% to 9.8% with 6,500 random traces, 10.8% with all 16,728, and 14.4% with a historical rule-selected subset (significant by paired bootstrap and McNemar), though the rule fails post-hoc validity checks and reverses on an unmatched 27B run; a 30-step online pilot shows skill-only training dropping GDPval-style score from 0.160 to 0.056 while mixing artifact tasks reaches 0.237.

Critical strengths

S1: Methodologically honest: off-policy is precisely defined as data-collection-level, and the simple SFT objective attributes gains to the supervision distribution.

S2: Clever two-branch design: feasibility diagnostics in professional work, controlled executable evidence in software artifacts, with the adjacency caveat stated explicitly.

S3: Rigorous pipeline auditing: preflight checks, solution-invariance criteria, task-level three-property audit, contamination screens, and a zero-repository-overlap subset.

Critical weaknesses

W1: Title-to-evidence mismatch: the controlled off-policy evidence is entirely on software patches, while the professional-artifact claims remain feasibility-level, as the authors concede.

W2: Small, partially unreliable professional data: only 22% of tasks retained, "scored attempts" do not imply usable artifacts, and rollout launch was not gated on integrity reports.

W3: Limited statistical power: one training seed per arm, token/cohort mismatch between Rule-selected and Random, equal-epoch (not equal-compute) Full, and no Base/Random control in the 27B check.

W4: Selection-bias risk in a headline result: the v1 gain is most plausibly a composition effect of the success gate, the rule fails post-hoc validity checks and reverses at 27B, yet it is presented prominently.

Questions for authors

See Weaknesses.

Reviewer_B4qVOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces WorkSynth, a pipeline for generating synthetic training data for agents that produce professional artifacts such as documents, spreadsheets, and presentations. The main idea is to use offline supervised fine-tuning on previously collected agent trajectories, potentially avoiding the costs of online reinforcement learning. The authors present a controlled study on SWE-bench Verified, where fine-tuning improves a 9B model, alongside a separate pipeline for generating professional-artifact tasks and an online GDPval-style pilot.

Critical strengths
  • Interesting and relevant problem. Improving agents' ability to produce complete, high-quality professional artifacts is an important problem that is relatively understudied compared with coding and reasoning benchmarks. The motivation for exploring offline training instead of expensive online RL is compelling, particularly given the complexity of evaluating long-running artifact-generation workflows.

  • Useful discussion of practical challenges. The paper provides interesting observations about the difficulties of constructing and evaluating professional-artifact tasks, including unreliable reward signals, missing deliverables, and overly restrictive evaluation criteria. I also appreciate the authors' transparency about the limitations of their experiments.

Critical weaknesses

I find the problem important and the motivation compelling, but the experimental evaluation falls short of supporting the paper's central claims. The strongest results are obtained on software engineering tasks using a relatively standard supervised fine-tuning approach, while the proposed professional-artifact training methodology remains largely untested. The acknowledged evaluation issues further weaken the conclusions.

Overall, this feels more like a promising research direction with preliminary experiments than a complete study. I would be substantially more positive if the authors demonstrated improvements on professional-artifact tasks using their proposed offline approach and isolated the benefits of the different supervision components.

In detail:

  • The main contribution is not experimentally demonstrated. My biggest concern is the disconnect between the paper's motivation and its experiments. The paper is framed around using off-policy synthetic data to improve professional-artifact agents, yet the only controlled offline training experiments are conducted on software engineering tasks. The professional-artifact experiments use online training or focus on data construction and evaluation diagnostics. Consequently, the paper does not actually demonstrate that the proposed approach improves the task domain it targets.

  • The proposed training methodology is not sufficiently evaluated. A central motivation is that different types of synthetic supervision, particularly critique-and-revision trajectories and genre-specific examples, can improve artifact quality. However, the paper does not compare these alternatives experimentally. Without these ablations, it is difficult to understand which aspects of WorkSynth matter or whether the proposed methodology offers advantages over standard supervised fine-tuning on successful trajectories.

  • The data-selection results are difficult to interpret. The rule-selected dataset performs better than random selection for the 9B model, but the selection criteria are confounded by factors such as trajectory success, length, and task composition. More concerningly, the ordering reverses in the 27B experiment, where training on the full dataset performs substantially better. This raises questions about whether the proposed selection strategy generalizes or simply benefits from selecting successful trajectories in the smaller-model setting. A success-only baseline would be particularly useful.

  • Problems with the professional-artifact evaluation. The authors acknowledge several issues with their evaluation pipeline, including criteria that incorrectly evaluate copied input attachments, inconsistent rubric judgments, and unmatched task populations across comparisons. These problems undermine the reliability of the professional-artifact results. In addition, only 29 tasks survive the trajectory-level audit, which seems quite limited for drawing broader conclusions about professional work.

  • Limited evidence for the claimed efficiency advantages. Avoiding online environment interactions during training is appealing, but generating synthetic trajectories still requires expensive teacher rollouts, sandbox execution, and evaluation. The paper does not provide an end-to-end computational cost comparison with online RL. As a result, it remains unclear whether WorkSynth offers meaningful efficiency advantages when data generation costs are included.

Questions for authors
  1. Can the authors provide any controlled offline fine-tuning results on the professional-artifact tasks themselves? This is essential to support the paper's central contribution.

  2. What are the total costs of generating and evaluating synthetic trajectories compared with online RL? Under what conditions would the proposed approach actually be more efficient?

Reviewer_QCYaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that post-training recipes built for short, verifiable outputs serve professional artifact work (reports, spreadsheets, decks, decision documents) poorly, and proposes off-policy supervision as a remedy: a teacher synthesizes briefs, reference deliverables, rubrics, and multi-policy rollouts, and a student is trained with assistant-token masked SFT on these frozen trajectories, with no sandbox or judge in the loop. The study is framed around three supervision axes (task formulation, artifact structure, and critique-and-revision feedback), and find that professional artifact agents may be limited less by a lack of generic reasoning ability than by insufficient exposure to the conventions, constraints, and revision practices of real-world knowledge work.

Critical strengths
  1. The paper is well organized and easy to follow. The illustrations are vivid and easy to understand.
  2. The method is simple and intuitive. It collects and synthesizes data off-policy and then performs post-training, following a standard and easily reproducible pipeline.
  3. The approach yields notable empirical improvements. For instance, the "All resolved" metric on SWE-bench increases from 8 to 72.
Critical weaknesses
  1. The method is relatively straightforward. Masked behavior cloning on agent trajectories is a well-established practice, and the evaluation design is also fairly standard. As a result, the technical novelty is limited.
  2. The experiments are insufficient. The paper compares against only a few baselines, omits comparisons with recent state-of-the-art methods, and evaluates on a limited set of datasets. Overall, the paper's novelty and contribution are not strong enough.
Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.