acceptodds
Under review as a conference paper at ICLR 2027

SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments

Abstract

Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging. Native application environments require substantial construction and integration effort, which limits application coverage and increases the risk of application-specific overfitting. Browser-hosted environments provide limited interaction depth and workflow complexity. These limitations motivate SoftwareGym, an automated pipeline for synthesizing GUI software environments with diverse interfaces, executable functionality, and runtime-verified behavior. It generates structurally diverse designs, realizes them as runnable applications, and uses GUI evaluation with iterative repair to ensure functional reliability. Using SoftwareGym, we synthesize 182 applications across twelve categories and achieve a mean GUI task pass rate of 91.5%. We construct SoftwareGym-Bench, a 120-task evaluation suite spanning the same twelve categories, and use trajectories collected from these environments for CUA training. The resulting model improves Pass@3 from 41.39% to 56.97% on the strictly out-of-domain OSWorld benchmark and achieves 40.0% on SoftwareGym-Bench. These results demonstrate the potential of synthesized software environments as scalable and transferable supervision for CUA training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer-Use Agents, GUI Agents, Environment Synthesis, Software Building

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_Vbt1OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents SoftwareGym, an automated pipeline for synthesizing executable desktop software environments for CUA training. The authors construct 182 applications across twelve categories and introduce SoftwareGym-Bench, comprising 120 tasks with programmatic evaluation. Supervised fine-tuning of Qwen3.5-9B on trajectories from these applications improves Pass@3 from 41.39% to 56.97% on the evaluated 244-task OSWorld subset and from 25.8% to 40.0% on SoftwareGym-Bench. Ablations examine structured interface design, GUI-guided repair, and training trajectories from synthesized applications.

Critical strengths
  1. The paper provides a pipeline to generate executable applications with persistent state and multi-step workflows, covering a significant number and categories of applications.
  2. The potential of generalization to other software domains is encouraging. Training only on SoftwareGym trajectories improves the same base model by 15.58pp on the evaluated OSWorld subset, with gains across all eight reported application domains (Table 1). This supports the generalization of their synthesized environments beyond their own benchmark.
  3. The GUI-guided repair pipeline on synthesized applications is interesting. The ablation also shows its strength. Both branches start from the same applications and undergo two repair rounds, with GUI feedback withheld from the source-only branch. Across 36 applications, GUI-guided repair reaches 83.1%, compared with 48.8% for source-only repair.
Critical weaknesses
  1. SoftwareGym substantially overlaps with existing environment-synthesis pipelines. InfiniteWeb [1] combines functional and visual diversity, specification-driven generation, test-driven repair, and transfer to OSWorld. GUI-GENESIS [2] synthesizes executable applications with state-based rewards and static/dynamic verification. ScaleWoB [3] additionally performs interactive GUI validation followed by environment repair and revalidation, with human failure triage. The paper should clarify which mechanisms constitute substantive advances over these predecessors.
  2. Desktop application synthesis and explicit structural diversity controls are potentially valuable contributions. However, the current ablations compare against simplified design and source-only repair without establishing an advantage over closely related synthesis pipelines. A focused comparison under matched functionality and training budgets would help determine whether these components improve transfer beyond the benefits of additional synthetic supervision.
  3. The SFT is conducted only on Qwen3.5-9B, leaving it unclear whether the benefits of synthesized environments extend to larger models or other model families.
  4. In Table 3, combining OSWorld and SoftwareGym data reduces OSWorld Pass@3 from 61.48% to 58.61% relative to OSWorld-only training. Meanwhile, SoftwareGym-Bench performance increases from 40.0% to 47.5% relative to SoftwareGym-only. The results fail to demonstrate consistent complementarity across both benchmarks, which the author claims at Line 471. Further justification is required.

References [1] Zhang et al. InfiniteWeb: Scalable Web Environment Synthesis for GUI Agent Training. 2026. https://arxiv.org/abs/2601.04126 [2] Cao et al. GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training. 2026. https://arxiv.org/abs/2602.14093 [3] Liu et al. ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis. 2026. https://arxiv.org/abs/2605.25160

Questions for authors
  1. What are the substantive methodological advances over existing environment-synthesis pipelines? Please provide a mechanism-level comparison with previous works, distinguishing new components from adaptations of existing approaches. Which specific limitations of these predecessors does SoftwareGym address?
  2. Could the authors provide a focused comparison against a closely related synthesis approach, controlling functionality, trajectory quantity, and training budget? A smaller experiment on representative application categories would help isolate these components’ benefits from those of additional synthetic supervision.
  3. Could the authors validate the approach on at least one larger model or another model family? If additional training is impractical, please clarify the scope of the generality claim and discuss whether the observed gains might depend on the chosen backbone.
4 comments
AuthorsOpenReview ↗

Response to Reviewer Vbt1 (1/3)

We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. In response, we conducted additional experiments and related work review. Our responses and additional results are detailed below.


Response to Weakness 1 and Question 1:

SoftwareGym substantially overlaps with existing environment-synthesis pipelines. InfiniteWeb [1] ... GUI-GENESIS [2] ... ScaleWoB [3]... The paper should clarify which mechanisms constitute substantive advances over these predecessors.

SoftwareGym differs from prior environment-synthesis pipelines in both the target environment substrate and the synthesis objective. While InfiniteWeb, GUI-GENESIS, and ScaleWoB primarily focus on web-based environments or task-specific web surrogates, SoftwareGym targets the synthesis of complete, runnable desktop applications for CUA training. The mechanism-level differences are summarized in Table A.

[Table A: Comparison of existing environment-synthesis methods across environment substrate, synthesis target, interaction scope, local OS/filesystem support, and validation mechanisms.]

MethodSubstrateSynthesis TargetInteraction ScopeNative OS / FilesystemValidation & Repair
InfiniteWeb [1]Browser-based WebMulti-page functional websitesMulti-page web workflowsNo native OS/FS interactionTDD, iterative auto-fix, dense rewards
GUI-GENESIS [2]Standalone Flask Web appTask-specific standalone applicationsTask-conditioned workflowsLocal database; no native OS/FS interactionCode-native rewards, static and Playwright verification
ScaleWoB [3]Backend-free WebFunctional app simulationsMobile, desktop, and automotive GUI profilesMock data and browser-local state; no native OS/FS interactionExecutable validators, task validation, human-in-the-loop repair
SoftwareGymNative desktopComplete runnable applicationsMultiple functions and workflowsLocal filesystem and persistent stateSource verification, GUI evaluation, iterative repair

What are the substantive methodological advances over existing environment-synthesis pipelines? Please provide a mechanism-level comparison with previous works, distinguishing new components from adaptations of existing approaches.

The methodological contributions of SoftwareGym lie in three aspects:

  1. Desktop application synthesis. We extend environment synthesis from web-based environments and task-specific surrogates to complete, runnable desktop applications with persistent local state and native and diverse GUI interactions.

  2. Quantitative and iterative diversity control. SoftwareGym quantifies software and interface diversity and uses these measurements to guide successive synthesis iterations. This provides a systematic, measurement-driven approach to diversity control rather than relying solely on manually specified variations.

  3. Closed-loop reliability assurance. We integrate functional specifications, source-level verification, GUI-based evaluation, and iterative repair to improve the functional correctness and reliability of generated applications.

Our contribution lies in integrating these mechanisms into a scalable pipeline for specification-driven, diversity-controlled desktop application synthesis.

Which specific limitations of these predecessors does SoftwareGym address?

Existing environment-synthesis approaches face two major limitations:

  1. native-application-based approaches incur substantial integration costs, limiting software coverage and increasing the risk of application-specific overfitting.

  2. browser-hosted environments offer scalable deployment and state control but are constrained by their shared browser/HTML/DOM substrate, which limits interaction depth and workflow complexity.

To address these limitations, SoftwareGym tackles three challenges:

  1. Interface and interaction diversity. Unconstrained generation within the same software category often produces similar layouts and interaction patterns. SoftwareGym introduces quantitative structural-diversity measurement and iterative design refinement to generate distinct software structures and interfaces.

  2. Functional reliability. Generated functions must remain usable throughout complete GUI workflows. SoftwareGym combines source-level verification, checkpoint-based GUI evaluation, and iterative repair to ensure functional correctness.

  3. Scalability. Applying diversity controls and reliability checks across a large software pool creates substantial verification overhead. SoftwareGym automates the design–implementation–evaluation pipeline, enabling batch synthesis with minimal human intervention.

AuthorsOpenReview ↗

Response to Reviewer Vbt1 (2/3)

Response to Weakness 2 and Question 2

... However, the current ablations compare against simplified design and source-only repair without establishing an advantage over closely related synthesis pipelines....Could the authors provide a focused comparison against a closely related synthesis approach, controlling functionality, trajectory quantity, and training budget? A smaller experiment on representative application categories would help isolate these components’ benefits from those of additional synthetic supervision.

We clarify that these methods target fundamentally different synthesis objectives. InfiniteWeb, GUI-GENESIS, and ScaleWoB focus on web-based environments or task-specific web surrogates, whereas SoftwareGym synthesizes complete, standalone desktop applications. Consequently, a direct comparison under matched functionality would require adapting these methods to a different environment substrate and synthesis objective.

To isolate the effect of interface diversity, we conducted a controlled Low-Diversity versus High-Diversity comparison. Low-Diversity contains 12 applications (one per category), while High-Diversity contains 182 applications across the same 12 categories. Both datasets were matched in size, category composition, difficulty, and quality, with identical training budgets.

Under matched training conditions, the results are shown in Table B. This demonstrates that greater software and interface diversity improves downstream transfer under the same data volume.

[Table B: Pass@1 comparison of Low-Diversity and High-Diversity training on the 244-task OSWorld application subset.]

DomainLow-DiversityHigh-DiversityQwen3.5-9B Base
Chrome17/46 (36.96%)15/46 (32.61%)11/46 (23.91%)
GIMP17/26 (65.38%)17/26 (65.38%)10/26 (38.46%)
LibreOffice Calc4/47 (8.51%)10/47 (21.28%)6/47 (12.77%)
LibreOffice Impress13/47 (27.66%)10/47 (21.28%)13/47 (27.66%)
LibreOffice Writer7/23 (30.43%)11/23 (47.83%)9/23 (39.13%)
Thunderbird8/15 (53.33%)7/15 (46.67%)7/15 (46.67%)
VLC5/17 (29.41%)6/17 (35.29%)6/17 (35.29%)
VS Code11/23 (47.83%)14/23 (60.87%)6/23 (26.09%)
Overall82/244 (33.61%)90/244 (36.89%)68/244 (27.87%)

Response to Weakness 3 and Question 3

The SFT is conducted only on Qwen3.5-9B, leaving it unclear whether the benefits of synthesized environments extend to larger models or other model families.....Could the authors validate the approach on at least one larger model or another model family? If additional training is impractical, please clarify the scope of the generality claim and discuss whether the observed gains might depend on the chosen backbone.

Following the reviewer's suggestion, we conducted an additional SFT experiment using Qwen3-VL-30B-A3B-Instruct, a substantially larger model from a different model family. We trained it using the same SoftwareGym training data as Qwen3.5-9B to examine whether the benefits of SoftwareGym generalize beyond the original backbone.

We evaluated the resulting model on the same 244 OSWorld application-domain tasks. The results are summarized in Table C:

The model achieves an overall average score of 48.32%, substantially exceeding the officially reported result of 30.60%. These results provide further evidence that the benefits of SoftwareGym training are not limited to Qwen3.5-9B and can extend to larger models from different model families.

[Table C: Performance of Qwen3-VL-30B-A3B-Instruct trained on SoftwareGym data across the eight OSWorld application domains.]

Application DomainTotal TasksPassPass RateAverage Score
Chrome462452.17%54.26%
GIMP261765.38%65.38%
LibreOffice Calc471429.79%29.79%
LibreOffice Impress471838.30%40.34%
LibreOffice Writer231252.17%56.51%
Thunderbird151173.33%73.33%
VLC17741.18%46.98%
VS Code231147.83%47.83%
Overall24411446.72%48.32%
AuthorsOpenReview ↗

Response to Reviewer Vbt1 (3/3)

Response to Weakness 4

In Table 3, combining OSWorld and SoftwareGym data reduces OSWorld Pass@3 from 61.48% to 58.61% relative to OSWorld-only training. Meanwhile, SoftwareGym-Bench performance increases from 40.0% to 47.5% relative to SoftwareGym-only. The results fail to demonstrate consistent complementarity across both benchmarks, which the author claims at Line 471. Further justification is required.

We clarify that mixed-source training should be assessed by overall performance across both benchmarks.

We have added the task-count-weighted Overall Pass@3 in Table D. The mixed model achieves the best overall result (54.95%), outperforming both OSWorld-only (52.19%) and SoftwareGym-only (51.38%) training, demonstrating complementary benefits from the two data sources.

[Table D: Pass@3 comparison of OSWorld-only, SoftwareGym-only, and combined training on OSWorld, SoftwareGym-Bench]

Training sourceOSWorld Pass@3SoftwareGym-Bench Pass@3Overall Pass@3 (364 tasks)
OSWorld-only61.48%33.3%52.19%
SoftwareGym-only56.97%40.0%51.38%
OSWorld + SoftwareGym58.61%47.5%54.95%

We thank the reviewer again for these valuable suggestions. We believe the additional experiments and clarifications substantially strengthen our empirical evidence and address the concerns raised. We hope our responses address your concerns and would appreciate it if you could raise your rating.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer Vbt1,

Thank you again for your thoughtful review. As the discussion period approaches its end, we would like to briefly follow up on our rebuttal.

We have provided additional experiments and clarifications addressing your concerns:

  • Novelty and Comparison: We clarified SoftwareGym's distinction from existing environment-synthesis and CUA training methods, including comparisons with related approaches.
  • Model Generalization: We conducted additional experiments with Qwen3-VL-30B-A3B, demonstrating the applicability of SoftwareGym to another model architecture.
  • Data Diversity and Complementarity: We provided controlled diversity experiments and mixed-source training ablations to validate their contributions.

We would greatly appreciate your feedback on whether these results address your concerns and would be grateful if you could consider increasing your rating.

Thank you for your time and consideration.

The Authors

Reviewer_JBcrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposed a CUA training system named SoftwareGym. The procedure includes designing software functionality and diverse interfaces, implementing the applications, and iteratively repairing them based on GUI test feedback.

Critical strengths
  • Training on synthetic data substantially improves performance on real applications. Automated data generation can also reduce data collection costs.
  • GUI-guided repair clearly outperforms repair based only on source code. This is a particularly valuable empirical finding.
Critical weaknesses
  • The diversity ablation only measures differences in interface structure. It does not test whether greater interface diversity improves downstream training results.
  • Mixed-source training uses roughly twice as much data as either single-source setting, making it difficult to separate data complementarity from increased data volume. The mixed model also performs worse than the native-data-only model on OSWorld.
Questions for authors
  • With the same data volume and training budget, how much improvement does interface diversity provide?
  • Does mixed-source training still offer an advantage when the total data volume is fixed?
  • Please clarify how training and evaluation tasks are split and deduplicated.
4 comments
AuthorsOpenReview ↗

Response to Reviewer JBcr (1/3)

We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. In response, we conducted additional controlled experiments on interface diversity and fixed-volume mixed-source training, reported the overall performance across both benchmarks, and further clarified our task-splitting and deduplication protocols. Our responses and additional results are detailed below.


Response to Weakness 1 and Question 1:

The diversity ablation only measures differences in interface structure. It does not test whether greater interface diversity improves downstream training results...With the same data volume and training budget, how much improvement does interface diversity provide?

To directly examine whether interface diversity improves downstream training, we additionally conducted a controlled Low-Diversity versus High-Diversity ablation:

We constructed two datasets: Low-Diversity (12 applications, one per category) and High-Diversity (182 applications across the same 12 categories). We matched their training volume, category composition, data splits, difficulty, quality distributions, and deduplication constraints, using identical backbones and training settings.

This controlled comparison isolates the effect of software and interface diversity from data volume and training budget. The results are shown in Table A.

Under matched training conditions, High-Diversity achieves 36.89% Pass@1, compared with 33.61% for Low-Diversity, yielding a 3.28-percentage-point improvement. This provides direct evidence that greater software and interface diversity improves downstream performance at a fixed data volume.

[Table A: Pass@1 comparison of Low-Diversity and High-Diversity training.]

DomainLow-DiversityHigh-DiversityQwen3.5-9B Base
Chrome17/46 (36.96%)15/46 (32.61%)11/46 (23.91%)
GIMP17/26 (65.38%)17/26 (65.38%)10/26 (38.46%)
LibreOffice Calc4/47 (8.51%)10/47 (21.28%)6/47 (12.77%)
LibreOffice Impress13/47 (27.66%)10/47 (21.28%)13/47 (27.66%)
LibreOffice Writer7/23 (30.43%)11/23 (47.83%)9/23 (39.13%)
Thunderbird8/15 (53.33%)7/15 (46.67%)7/15 (46.67%)
VLC5/17 (29.41%)6/17 (35.29%)6/17 (35.29%)
VS Code11/23 (47.83%)14/23 (60.87%)6/23 (26.09%)
Overall82/244 (33.61%)90/244 (36.89%)68/244 (27.87%)

AuthorsOpenReview ↗

Response to Reviewer JBcr (2/3)

Response to Weakness 2 and Question 2

The mixed model also performs worse than the native-data-only model on OSWorld.

We clarify that mixed-source training should be assessed by overall performance across both benchmarks.

We have added the task-count-weighted Overall Pass@3 in Table B. The mixed model achieves the best overall result (54.95%), outperforming both OSWorld-only (52.19%) and SoftwareGym-only (51.38%) training, demonstrating complementary benefits from the two data sources.

[Table B: Pass@3 comparison of OSWorld-only, SoftwareGym-only, and combined training on OSWorld, SoftwareGym-Bench]

Training sourceOSWorld Pass@3SoftwareGym-Bench Pass@3Overall Pass@3 (364 tasks)
OSWorld-only61.48%33.3%52.19%
SoftwareGym-only56.97%40.0%51.38%
OSWorld + SoftwareGym58.61%47.5%54.95%

Mixed-source training uses roughly twice as much data as either single-source setting, making it difficult to separate data complementarity from increased data volume.... Does mixed-source training still offer an advantage when the total data volume is fixed?

To address this concern, we conducted additional controlled experiments to examine whether mixed-source training retains its advantage under matched data-volume and training-budget conditions.

We compare three training-data configurations:

SoftwareGym-only (1×), using only SoftwareGym trajectories.

Half-Mix (1×), combining SoftwareGym and OSWorld trajectories at a fixed total volume.

Full Union (2×), combining the complete datasets from both sources.

All three SFT models use the same backbone, data preprocessing, split protocol, and optimization settings; only the data-source composition and total data volume differ.

Results are shown in Table C.These results demonstrate that mixed-source training provides still offer an advantage when the total data volume is fixed.

[Table C: Pass@1 comparison of Qwen3.5-9B models trained with different data-source configurations and training volumes.]

DomainQwen3.5-9B BaseSoftwareGym-only (1×)Half-Mix (1×)Full Union (2×)
Chrome11/46 (23.91%)24/46 (52.17%)27/46 (58.70%)23/46 (50.00%)
GIMP10/26 (38.46%)17/26 (65.38%)17/26 (65.38%)17/26 (65.38%)
LibreOffice Calc6/47 (12.77%)11/47 (23.40%)12/47 (25.53%)14/47 (29.79%)
LibreOffice Impress13/47 (27.66%)19/47 (40.43%)17/47 (36.17%)21/47 (44.68%)
LibreOffice Writer9/23 (39.13%)11/23 (47.83%)13/23 (56.52%)11/23 (47.83%)
Thunderbird7/15 (46.67%)10/15 (66.67%)8/15 (53.33%)9/15 (60.00%)
VLC6/17 (35.29%)5/17 (29.41%)7/17 (41.18%)9/17 (52.94%)
VS Code6/23 (26.09%)13/23 (56.52%)12/23 (52.17%)10/23 (43.48%)
Overall68/244 (27.87%)110/244 (45.08%)113/244 (46.31%)114/244 (46.72%)
AuthorsOpenReview ↗

Response to Reviewer JBcr (3/3)

Response to Question 3

Please clarify how training and evaluation tasks are split and deduplicated.

We adopt a unified data construction pipeline to ensure task-level separation between training and evaluation, remove redundant tasks, and retain high-quality training trajectories. All benchmark evaluation tasks and their variants are excluded from training, and no exact evaluation task, trajectory, or initial-state instance is reused. The pipeline consists of the following three stages:

  1. Task splitting. We normalize task descriptions by masking instance-specific values such as file names, dates, coordinates, and text contents, while preserving application context, initial-state configuration, ordered operations, and completion criteria. Tasks sharing the same normalized functional structure are assigned to the same partition. The training data are then deterministically split into train, validation, and test sets using a 90/5/5 ratio. We verify that no functional task groups or individual trajectory attempts overlap across partitions.

  2. Deduplication. We perform deduplication in two stages: (i) exact deduplication based on normalized task descriptions, application types, initial-state configurations, and resource types; and (ii) semantic deduplication based on normalized requests, abstracted action sequences, declared operations, and observed interaction summaries, followed by cluster-level consistency checks. Two trajectories are considered duplicates when their ordered functional operations and completion requirements remain identical after masking instance-specific values.

  3. Trajectory selection. We retain at most two high-quality trajectories for each functional task, prioritizing complementary sources when available. The detailed quality assessment and trajectory filtering procedures are described in Appendix F.


We thank the reviewer again for these valuable suggestions. We believe the additional experiments and clarifications substantially strengthen our empirical evidence and address the concerns raised. We would appreciate it if you could raise your rating.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer JBcr,

Thank you again for your thoughtful review. As the discussion period approaches its end, we would like to briefly follow up on our rebuttal.

We have conducted additional controlled experiments addressing your key concerns:

  • Interface Diversity: We compared Low-Diversity (12 applications) and High-Diversity (182 applications) under matched training conditions, demonstrating the benefits of greater software diversity.
  • Data Complementarity: We conducted fixed-volume mixed-source experiments, showing that mixing SoftwareGym and OSWorld data retains its advantage under the same training budget.
  • Data Splitting and Deduplication: We further clarified our task-splitting and deduplication protocols.

We would greatly appreciate your feedback on whether these experiments address your concerns and would be grateful if you could consider increasing your rating.

Thank you for your time and consideration.

The Authors

Reviewer_RPsKOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents SoftwareGym, a multi-agent pipeline for creating desktop applications as training environment for compute-use agents. The pipeline has three stages: 1) Design: A functional spec and several UI blueprints 2) Implementation: A coder agent builds the app and DevCoder verifies its functionality against manual 3) Evaluation and repair: Auto-generated GUI test cases are run by agent. Only applications that pass both source-level and GUI verification enter the library.

Using the pipeline the authors generate 182 runnable applications across 12 categories (pass rate 91.5%) and they constructed SoftwareGym-Bench which has 120-tasks. They collected SoftwareGym trajectories for SFTQwen3.5-9B which improves Pass@3 from 41.39% to 56.97% on a 244-task OSWorld subset and from 25.83% to 40.0% on SoftwareGym-Bench.

Critical strengths

A well-engineered pipeline: Paper connects application design, implementation, GUI testing, repair, and task generation. In the paired 36-application experiment, GUI-guided repair reaches 83.1%, compared with 48.8% for source-only repair from the same starting applications. The reported difference has a 95% confidence interval of 28.2–41.0 percentage points. The repair ablation (Fig. 7b, App. C.2) is clean and informative.

Real transfer signal: A model trained exclusively on SoftwareGym data improves OSWorld Pass@3 by 15.58 points without any OSWorld applications, tasks, or trajectories.

Critical weaknesses

Reported software reliability is measured on the repair suite: After the initial implementation, GUI cases are generated which were reused to guide repairs (section 3.2). Hence the final 91.5% measures performance on these same cases, leaving correctness on new workflows uncertain. SoftwareGym-Bench adds manually verified tasks on twelve selected applications, but does not independently validate reliability across all 182 applications. This matters because reliable environments are central to the proposed training pipeline.

Questions for authors

Reliability clarification: Have the generated applications been tested on fresh workflows beyond the cases used during repair?

Evaluation-protocol clarification: How were the 244 OSWorld tasks selected?

4 comments
AuthorsOpenReview ↗

Response to Reviewer RPsK (1/3)

We appreciate the reviewer’s careful reading of our manuscript and the opportunity to clarify the scope of our reliability evaluation. Specifically, we clarify the purpose of the fixed-suite evaluation, report the fresh-workflow evaluation separately, and provide complete reliability results for all 182 generated software variants.

Response to Weakness 1 and Question 1

After the initial implementation, GUI cases are generated which were reused to guide repairs (section 3.2). Hence the final 91.5% measures performance on these same cases, leaving correctness on new workflows uncertain.

We deliberately retained the same GUI cases across successive repair rounds for three reasons:

  1. To ensure consistent evaluation difficulty and task requirements across versions.
  2. To verify whether previously identified failures were actually resolved.
  3. To track improvements easily across successive repair rounds.

Using different GUI cases across rounds would:

  1. Confound repair improvements with changes in task difficulty.
  2. Prevent direct comparison of previously observed failures.
  3. Make it unclear whether performance changes result from repairs or differences in evaluation cases.

Reliability clarification: Have the generated applications been tested on fresh workflows beyond the cases used during repair?

Following the reviewer's suggestion, we conducted an additional evaluation using newly generated evaluation tasks at each round to assess whether the improvements generalize to unseen tasks and workflows.

Specifically, we conducted three rounds of GUI evaluation on two applications, ForumAPP and LocalRichEditor, using a different set of evaluation tasks in each round. ForumAPP and LocalRichEditor. Before evaluating each version, we generated a new set of GUI evaluation tasks that had not been used in any previous round. Thus, each version was evaluated on fresh tasks. Qwen3.8-Max was used for all three model-dependent stages: evaluation-task generation, GUI execution, and result judging.

The results are summarized in Table A:

[Table A: Pass rates across three successive software versions evaluated on independently generated fresh GUI tasks.]

Applicationv1v2v3
ForumAPP25.00% (5/20)30.00% (6/20)57.89% (11/19)
LocalRichEditor55.00% (11/20)60.00% (12/20)75.00% (15/20)

.... but does not independently validate reliability across all 182 applications. This matters because reliable environments are central to the proposed training pipeline.

To address this concern, we provide the complete repair results for all 182 generated software variants, including their performance at each repair round and their final available versions, as summarized in Table B.

The aggregate pass rate steadily increases across repair rounds. More importantly, across the final available versions of all 182 variants, the average pass rate reaches 90.3%. These results demonstrate that the final software variants retained for training are of high quality.

AuthorsOpenReview ↗

Response to Reviewer RPsK (2/3)

[Table B. Pass Rates (%)]

182 software variants across 12 applications (13–19 variants each); all values are pass rates (%), rounded to the nearest whole number. Columns 01–19 indicate the variant ID within each application. V1, V2, and V3 represent the three evaluation versions. — means not run, N/A means no evaluation tasks were available, and / means no such variant.

ApplicationStage01020304050607080910111213141516171819
Browser applicationV109472072896580766818696//////
V2471008395941009487100100657553//////
V394—9489100—9487——829488//////
Chat applicationV171086539267N/A752973947983//////
V236818687698788508680888994//////
V3861001009392879488868710084100//////
Code editorV14675796969697657695779730677260///
V2928110092888894939493937377878980///
V38581—1008110010093941001008792936753///
Forum applicationV1678179687667826530402530425355////
V283941009510094949075855680899470////
V310094N/A100—941009085908170848890////
Rich-text editorV15081697507573088643747535565N/A25//
V271691009464818775889384885980761288//
V38694—949394939494100891008895948881//
Mail clientV18156442573447369626980506580/////
V2948875N/A8788801008894901007580/////
V3949494—1009487—888895—9095/////
Music playerV1658980569069069623860405555/////
V29089857285757394857575657580/////
V360100808395948788928190858075/////
PDF toolV167616182615025948765592528385347506545
V294839488897838888794768822888284949085
V3100100941009494818893100888183888889899085
Image editorV181719371627960100073676709473////
V29493938677100100—60938987769493////
V31001009386100———8710094808810093////
Presentation editorV1504760811907569297344826500////
V208287888857818150808388754256////
V3888287888886888886939494958994////
Spreadsheet editorV1547796465630413356N/A76N/A565947///
V292869386808472767856N/A8268947684///
V3100931008690100838894818910089838279///
Video playerV1073857585647960N/A58678768380////
V2679110083100919380N/A8375808183100////
V38391—92—7329908383928010083—////
AuthorsOpenReview ↗

Response to Reviewer RPsK (3/3)

Response to Question 2

Evaluation-protocol clarification: How were the 244 OSWorld tasks selected?

The original evaluation covered 244 tasks across the eight OSWorld application domains (Chrome, GIMP, LibreOffice Calc, LibreOffice Impress, LibreOffice Writer, Thunderbird, VLC, and VS Code), focusing on standalone software interactions aligned with SoftwareGym's scope.

Following the reviewer's suggestion, we have now additionally evaluated 24 OS tasks and 93 multi-application tasks, expanding the evaluation to 361 tasks. This covers the full OSWorld test set except for the eight tasks requiring Google Drive integration, which is unavailable in our evaluation environment.

The improvement also extends to the OS and multi-application subsets, demonstrating transfer to broader OSWorld workflows beyond the standalone software environments used for training.

[Table C: Pass@1 results of Kimi 2.6, Qwen3.5-9B, and SoftwareGym-trained Qwen3.5-9B on the expanded OSWorld evaluation.]

ModelChromeGIMPLibreOffice CalcLibreOffice ImpressLibreOffice WriterThunderbirdVLCVS CodeOSmulti-appsOverall (361 tasks)
Kimi 2.624/46 (52.17%)9/26 (34.62%)31/47 (65.96%)32/47 (68.09%)14/23 (60.87%)13/15 (86.67%)9/17 (52.94%)10/23 (43.48%)19/24 (79.17%)25/93 (26.88%)186/361 (51.52%)
Qwen3.5-9B11/46 (23.91%)10/26 (38.46%)6/47 (12.77%)13/47 (27.66%)9/23 (39.13%)7/15 (46.67%)6/17 (35.29%)6/23 (26.09%)8/24 (33.33%)9/93 (9.68%)85/361 (23.55%)
SoftwareGym-trained Qwen3.5-9B24/46 (52.17%)17/26 (65.38%)11/47 (23.40%)19/47 (40.43%)11/23 (47.83%)10/15 (66.67%)5/17 (29.41%)13/23 (56.52%)13/24 (54.17%)11/93 (11.83%)134/361 (37.12%)

We thank the reviewer again for these valuable suggestions. We believe the additional experiments and clarifications substantially strengthen our empirical evidence and address the concerns raised. We hope our responses address your concerns and would appreciate it if you could raise your rating.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer RPsK,

Thank you again for your thoughtful review. As the discussion period approaches its end, we would like to briefly follow up on our rebuttal.

We have conducted additional evaluations to address your concerns:

  • Reliability Evaluation: We conducted three rounds of GUI evaluation using newly generated tasks in each round, demonstrating improved reliability across iterative repairs.
  • Comprehensive Results: We provided complete reliability results for all 182 generated applications, demonstrating the high reliability of the entire software library.
  • OSWorld Coverage: We expanded the evaluation to 361 tasks, covering all supported OSWorld tasks.

We would greatly appreciate your feedback on whether these additional results address your concerns and would be grateful if you could consider increasing your rating.

Thank you for your time and consideration.

The Authors

Reviewer_oDgaOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. What do the reference pipelines score when given the same budget and checkpoint-selection rule as the agent submissions?
  2. Do the system ranking and the learning-level vs. run-level gap hold after removing RAGEN and OWL, or under repeated runs?
  3. How were the normalization anchors chosen (e.g., for DDPO), and were high-scoring outputs inspected for reward over-optimization?"
Critical strengths
  • Using GUI-level feedback to repair synthesized applications is well motivated, and the ablation against source-only repair shows a large and consistent improvement in functional reliability.
  • The comparison of training data sources at equal size is informative and helps place synthesized environments relative to real-application trajectories."
Critical weaknesses
  • The "strictly out-of-domain" framing is hard to accept, since most synthesized categories map closely onto the OSWorld test domains (e.g., spreadsheet, code editor, image editor). Training only on non-aligned categories, or holding out an aligned category, would show whether the gains reflect transfer rather than category overlap.
  • The source of the gains is not yet isolated: the model used to collect training trajectories is not disclosed, and results are Pass@3 on a subset of OSWorld without confidence intervals. Naming the teacher, adding a teacher-controlled baseline, and reporting Pass@1 on the full benchmark could make this a strong paper.
Questions for authors
  1. Which model collected the training trajectories, and how does the same teacher perform with an equal number of trajectories from another environment source?
  2. Where do the OSWorld-only training trajectories come from, and do they overlap with the evaluation tasks or their variants?
  3. What are the results on the full OSWorld benchmark, including multi-app tasks, with Pass@1 and confidence intervals?"
3 comments
AuthorsOpenReview ↗

Response to Reviewer oDga (1/2)

We appreciate the reviewer’s thoughtful reading and the questions raised. We have clarified the teacher model, added experiments, and expanded the evaluation results as follows.

Response to Weakness 1:

The "strictly out-of-domain" framing is hard to accept...or holding out an aligned category, would show whether the gains reflect transfer rather than category overlap.

To address this concern, we conducted a strict out-of-domain experiment at the application-category level.

Specifically, we removed all Code Editor trajectories from the SoftwareGym training data and retrained the model using the same training procedure.

We then evaluated the resulting model on the 23-task VS Code subset. The results are shown in Table A.

[Table A: Results of the Code-Editor-held-out experiment]

ModelSuccessful tasksSuccess rate on VS Code
Base Qwen3.5-9B6/2326.09%
Full SoftwareGym model13/2356.52%
Model trained with the Code Editor category held out12/2352.17%

The Code-Editor-held-out model still substantially outperforms the base model. This result indicates that the improvement cannot be attributed solely to training on the same application category as the evaluated domain.


Response to Weakness 2 and Question 3:

Naming the teacher, adding a teacher-controlled baseline, and reporting Pass@1 on the full benchmark could make this a strong paper.......What are the results on the full OSWorld benchmark, including multi-app tasks, with Pass@1 and confidence intervals?

Following your suggestions, we conducted additional OSWorld evaluations:

  1. Teacher evaluation. We evaluated Kimi 2.6, the teacher model used for trajectory collection.
  2. Pass@1 evaluation. We additionally report Pass@1 results for all evaluated models on the expanded benchmark.
  3. OS and multi-application subsets. We extended the evaluation of both Qwen3.5-9B and SoftwareGym-trained Qwen3.5-9B to the OS and multi-application subsets, beyond the original application-domain tasks.

We report the Pass@1 results for all three models on the expanded evaluation in Table B.

The expanded evaluation covers 361 tasks: 244 application-domain tasks, 24 OS tasks, and 93 multi-application tasks. We exclude the OSWorld tasks that explicitly require Google Drive integration because this functionality is unavailable in our evaluation environment.

The improvement extends beyond the original application-domain evaluation to both OS tasks and multi-application tasks, demonstrating transfer to broader OSWorld workflows.

[Table B: Pass@1 results of Kimi 2.6, Qwen3.5-9B, and SoftwareGym-trained Qwen3.5-9B]

ModelChromeGIMPLibreOffice CalcLibreOffice ImpressLibreOffice WriterThunderbirdVLCVS CodeOSmulti-appsOverall (361 tasks)
Kimi 2.624/46 (52.17%)9/26 (34.62%)31/47 (65.96%)32/47 (68.09%)14/23 (60.87%)13/15 (86.67%)9/17 (52.94%)10/23 (43.48%)19/24 (79.17%)25/93 (26.88%)186/361 (51.52%)
Qwen3.5-9B11/46 (23.91%)10/26 (38.46%)6/47 (12.77%)13/47 (27.66%)9/23 (39.13%)7/15 (46.67%)6/17 (35.29%)6/23 (26.09%)8/24 (33.33%)9/93 (9.68%)85/361 (23.55%)
SoftwareGym-trained Qwen3.5-9B24/46 (52.17%)17/26 (65.38%)11/47 (23.40%)19/47 (40.43%)11/23 (47.83%)10/15 (66.67%)5/17 (29.41%)13/23 (56.52%)13/24 (54.17%)11/93 (11.83%)134/361 (37.12%)

Response to Question 1:

Which model collected the training trajectories?

We used Kimi 2.6 to collect the training trajectories.

How does the same teacher perform with an equal number of trajectories from another environment source?

We have already conducted the requested equal-data comparison in Section 4.3 (Ablation Studies), comparing models trained on SoftwareGym-only and OSWorld-only trajectories collected by the same teacher.

All training trajectories were collected by Kimi 2.6. The two datasets were matched in scale and underwent the same preprocessing. All models were initialized from the same Qwen3.5-9B base model. The complete data-construction and training details are provided in Appendix C.3.

For your reference, we provide the results here in Table C:

[Table C: Pass@3 comparison of models trained on different environment sources under matched data scale]

Training sourceOSWorld Pass@3SoftwareGym-Bench Pass@3Overall Pass@3 (364 tasks)
OSWorld-only61.48%33.3%52.19%
SoftwareGym-only56.97%40.0%51.38%
OSWorld + SoftwareGym58.61%47.5%54.95%

AuthorsOpenReview ↗

Response to Reviewer oDga (2/2)

Response to Question 2:

Where do the OSWorld-only training trajectories come from?

For the OSWorld-only condition, we used native OSWorld applications as the environment source and collected GUI interaction trajectories following the same task-generation and trajectory-collection procedure described in Section 3.3.

Do they overlap with the evaluation tasks or their variants?

We clarify that the training data do not overlap with the OSWorld evaluation tasks or their variants for two reasons:

  1. The training tasks were independently generated based on the actual software capabilities, rather than derived from or designed to imitate OSWorld evaluation tasks.
  2. The benchmark evaluation tasks, their variants, and their corresponding trajectories were not reused during training.

Although some tasks may share broad functional objectives because they involve the same software capabilities, they do not share the same task instances or trajectories.


We thank the reviewer again for these valuable suggestions. We hope our responses address your concerns and would appreciate it if you could raise your rating.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer oDga,

Thank you again for your thoughtful review. As the discussion period approaches its end, we would like to briefly follow up on our rebuttal.

We have provided additional experiments and clarifications addressing your concerns:

  • Transfer Generalization: We conducted a category-holdout experiment, showing that excluding all Code Editor training data still yields substantial improvements (52.17% vs. 26.09%).
  • Training Provenance: We clarified the teacher model (Kimi 2.6) and controlled training data construction.
  • Evaluation Completeness: We expanded OSWorld evaluation to 361 tasks, excluding only 8 unsupported Google Drive tasks.

We would greatly appreciate your feedback on whether these results address your concerns and would be grateful if you could consider increasing your rating.

Thank you for your time and consideration.

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Area Chair,

Thank you for considering our submission, SoftwareGym. We sincerely appreciate the reviewers' thoughtful feedback and the opportunity to further strengthen our work.


Strengths recognized by the reviewers. All reviewers consistently recognize the effectiveness of our software synthesis pipeline. Reviewer RPsK describes SoftwareGym recognizes a "real transfer signal": a model trained exclusively on SoftwareGym data achieves substantial improvements on OSWorld without using any OSWorld applications, tasks, or trajectories during training. Reviewer oDga highlights that GUI-level feedback that the equal-sized training-source comparison is informative. Reviewer Vbt1 further acknowledges the potential for generalization to other software domains. Reviewer JBcr recognizes that training on synthetic data substantially improves performance on real applications.


Key points of our responses:

  1. SoftwareGym demonstrates strong transfer to held-out application categories. We conducted a Code-Editor-held-out experiment, where the resulting model achieves 52.17% on VS Code, substantially outperforming the base model (26.09%).

  2. We demonstrate that greater software and interface diversity leads to better downstream performance under a fixed data budget. High-Diversity training achieves 36.89% Pass@1, outperforming Low-Diversity training (33.61%) under matched training conditions.

  3. Mixed-source training retains its advantage under a fixed data budget. Fixed-volume Half-Mix achieves 46.31% Pass@1, outperforming SoftwareGym-only training (45.08%). The full mixed-source model also achieves the best overall Pass@3 across both benchmarks (54.95%).

  4. SoftwareGym's training benefits extend to larger models and different model families. We conducted additional SFT experiments using Qwen3-VL-30B-A3B-Instruct, achieving a 48.32% average score on the 244-task OSWorld application evaluation.

  5. We expanded the OSWorld evaluation and reported teacher and student performance. We evaluated Kimi 2.6 (teacher), Qwen3.5-9B (base), and SoftwareGym-trained Qwen3.5-9B on 361 OSWorld tasks, covering application, OS, and multi-application tasks, excluding eight Google Drive tasks unavailable in our environment.

  6. We clarified the software reliability evaluation and provided additional evidence of software quality. We explained why fixed GUI test suites are retained across repair rounds to ensure comparable evaluation difficulty and track repair progress. Separately, we evaluated fresh tasks at each round on two applications and provided complete iteration results for all 182 generated applications, with a 90.3% average pass rate across their final available versions.

  7. We clarified the limitations addressed by SoftwareGym and highlighted its methodological innovations over prior work. We added a mechanism-level comparison with InfiniteWeb, GUI-GENESIS, and ScaleWoB, highlighting our focus on complete, runnable desktop applications, native GUI interactions, local filesystem support, quantitative diversity control, and closed-loop reliability assurance.

  8. We clarified training data construction and deduplication. We detailed task-level splitting, exact and semantic deduplication, and trajectory selection, ensuring that benchmark evaluation tasks and their variants are not reused during training.


We would greatly appreciate the Area Chair's help in encouraging further reviewer engagement while the discussion period remains open. We hope these additional experiments and clarifications will be helpful in informing the final assessment of our work.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.