- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Spreadsheet-RL combines automatically constructed initial/final workbook pairs, a spreadsheet-native interaction harness, an asynchronous Excel-backed verifier, and GRPO post-training. It also introduces the 1,660-task Domain-Spreadsheet benchmark.
Critical strengths
The environment and data pipeline address substantive obstacles to spreadsheet-agent training: persistent workbook state, formula recalculation, and scalable outcome verification. Constructing 5,928 training tasks and documenting an asynchronous Excel verifier constitutes a useful contribution beyond applying an RL objective to an existing text-only environment.
Critical weaknesses
The evaluation contract is inconsistent across the paper. Sections 3.3.1 and 5.1 describe formula- or structure-level checks, whereas Appendix A.11 specifies comparison of normalized, post-recalculation cell values only, including rounding numbers to two decimal places. Under the latter rule, a hard-coded value can match an oracle formula, and formatting-only requirements may not be represented by the score. The human audit establishes oracle validity and perceived metric suitability, not the evaluator’s error rate on model outputs, as the authors acknowledge. This uncertainty matters for interpreting the reported gains as successful workbook editing rather than value matching.
Questions for authors
In an audit of existing pre- and post-RL outputs, how often does the evaluator accept a workbook that violates the instruction, or reject a valid alternative? Does either error rate change after RL?
What checks exclude shared source threads, duplicate workbooks, and closely related template variants across training data, SpreadsheetBench, and Domain-Spreadsheet?
How many independent RL runs support the reported gains? What uncertainty is associated with the difference between the harness-only and harness-plus-RL conditions?
4 comments
Response to Reviewer scXg (2/2)
Q2 (deduplication across datasets). Before building the training set, we removed 26 crawled threads that overlap with SpreadsheetBench and 188 near-duplicate threads within the crawl; the revision documents this step in Section 5.1 and Appendix A.1.
- Training data vs. SpreadsheetBench. All crawled ExcelForum threads were compared with SpreadsheetBench by identical attachment files, overlap of distinctive cell values, shared word sequences in the task description, and first-post text similarity. The 26 removed threads are the 14 SpreadsheetBench source threads in our crawl (shared threads can only fall between 2024-01-01, the start of our crawl, and 2024-03-19, SpreadsheetBench's latest source post) and 12 near-duplicates of SpreadsheetBench tasks. None of the 5,928 training tasks comes from them. Because all crawled threads were compared, the content checks also cover a SpreadsheetBench workbook reposted in a later thread.
- Within the training data. Threads with identical attachments or near-identical first posts were grouped, and only the earliest of each group was kept.
- Template variants. These are covered from two sides: the cell-overlap check catches variants that share much of their content with a SpreadsheetBench workbook, and the task-description and first-post checks catch the same request posed on a different workbook.
- Domain-Spreadsheet. Its seeds are domain concepts and professional templates from certification-aligned sources (Section 4), not forum threads, so it shares no source threads with the other two datasets.
Q3 (number of RL runs; uncertainty of the gain). Four independent RL runs support the main gain, and for each of the three new runs the 95% confidence interval of the before-vs-after-RL difference excludes zero (new Appendix A.22). We trained the main configuration (Qwen3-4B-Thinking-2507 with harness and tools) three more times with different seeds, which set the data order and rollout sampling (with deterministic kernels), and report every run at its final step 60, without checkpoint selection; the RL-only and Qwen3-8B configurations, which support secondary results, have one run each. The pre-RL condition (harness and tools) is step 0 of the same run (19.3% in Table 1 for the original run), so each gain is paired within a run. SpreadsheetBench Pass@1 (%):
| Run | Step 0 (before RL) | Step 60 (after RL) | Gain | 95% CI of gain |
|---|---|---|---|---|
| Original (paper) | 19.3 | 23.4 | +4.1 | – |
| Seed 1 | 19.4 | 23.9 | +4.5 | [+2.9, +6.2] |
| Seed 2 | 18.9 | 22.6 | +3.7 | [+2.3, +5.2] |
| Seed 3 | 19.2 | 24.6 | +5.4 | [+3.6, +7.1] |
| Mean (SD), 4 runs | 19.2 (0.2) | 23.6 (0.8) | +4.4 (0.7) |
The intervals use standard errors clustered by task (t distribution, 911 degrees of freedom), since the three test cases of each of SpreadsheetBench's 912 tasks differ in their workbook data and are not independent. The original run's per-test-case results were not retained, so it has no interval. On Domain-Spreadsheet, the gain is +8.8 points in the original run and +9.7, +11.6 and +14.0 in the three new runs (mean +11.0, SD 2.3).
Q4 (personal and confidential information; redistribution). We agree that the data must be screened at the workbook level before release, and our release process does so. Beyond the URL redaction in thread text (Ethics Statement), we have scanned all 5,928 training tasks at the workbook level. The scan covers the instruction and all text in the input and target workbooks that the model can read, including cell text, string constants in formulas, comments and their authors, headers and footers, document properties such as author and company, and sheet, table and column names. We use Presidio with spaCy named-entity recognition and pattern recognizers for contact details, addresses, and ID, card and bank numbers. Flagged items are mainly person names, most of them in document and comment author fields, plus e-mail addresses, phone numbers, street addresses and IBAN-like numbers in a few tasks; all personal and confidential information is removed or replaced before release. Domain-Spreadsheet does not use forum attachments (Q2), and its seed data contain no personal or confidential information. The revision describes this screening in the Ethics Statement.
For redistribution, we follow the practice of SpreadsheetBench, which also builds on forum workbooks (Appendices A.3 and A.4 of its paper): our crawler strictly respects ExcelForum's robots.txt, and we will release only the processed, screened tasks, not the raw crawled threads or attachments.
We hope these responses have addressed the reviewer's concerns, and we are happy to answer any further questions during the discussion period.
Thank you for the evaluator clarification, output audit, and additional training runs. These substantially address my concerns and strengthen the evidence for RL gains. The remaining issue is comparability with prior results under different evaluators. Please identify which comparisons use the same scoring rules and, where possible, report official values-only scores on the same outputs. Redistribution terms should also be documented as a data-release clarification, rather than a methodological weakness. I maintain 3 (weak acceptance), with increased confidence in the results. If there are subdivisions such as 3.5 or 7, I will consider raising the score.
Response to Reviewer scXg’s follow-up
We thank Reviewer scXg for the follow-up and are glad that the evaluator clarification, the output audit, and the additional training runs address the main concerns.
Comparability with prior results. Under SpreadsheetBench's official evaluator, which compares values only, the RL gain grows in every run, and the comparison with reported closed-source results does not depend on our stricter checks.
Which comparisons share scoring rules.
- Our evaluator (values, formula presence, formatting; Section 3.3.1): all rows marked Ours in Table 1 except o3 with our harness and tools, i.e., the Qwen3 baselines, the base-to-RL progression of Qwen3-4B-Thinking-2507, RL-only, and Qwen3-8B, as well as the seed runs (Appendix A.22) and the output audit (Appendix A.23). Every before-vs-after-RL comparison therefore uses the same rules.
- Official values-only evaluator: o3 with our harness and tools (52.2%) and the rescored runs below. The revision states this in the Table 1 caption and Appendix A.16.
- Reported by their sources in their own environments: the other closed-source rows (GPT-4o, o3, ChatGPT agent, Claude Files Opus 4.1, Copilot Agent Mode), which we cannot rescore on the same outputs.
Official values-only scores on the same outputs. For the three runs that we newly trained, we rebuilt each output workbook by replaying the trajectory's tool calls on the test case's input workbook; the rebuilt outputs reproduce each run's scores under our evaluator exactly. We then scored all 2,726 test cases with the official evaluator. SpreadsheetBench Pass@1 (%):
| Run | Step 0 (before RL): ours / official | Step 60 (after RL): ours / official | Gain: ours / official |
|---|---|---|---|
| Seed 1 | 19.4 / 20.2 | 23.9 / 25.5 | +4.5 / +5.3 |
| Seed 2 | 18.9 / 19.6 | 22.6 / 23.8 | +3.7 / +4.2 |
| Seed 3 | 19.2 / 19.9 | 24.6 / 26.4 | +5.4 / +6.5 |
| Mean, 3 runs | 19.2 / 19.9 | 23.7 / 25.2 | +4.5 / +5.3 |
Because the official evaluator accepts every output that ours accepts, all scores rise. After RL, the 4B agent reaches 23.8–26.4% under the official rule, so its comparison with the reported o3 (23.3%) and GPT-4o (18.4%) results does not rest on our stricter checks.
Redistribution terms. We agree that these belong in the data-release documentation, and the release will document them.
We hope this response addresses the remaining concerns, and we are happy to answer any further questions during the discussion period.
Response to Reviewer scXg (1/2)
We thank Reviewer scXg for the detailed comments and for highlighting our paper's contributions in a realistic environment and data pipeline and scalable Excel-based verification. Below, we provide a detailed response to each concern.
W1 & Q1 (evaluation contract; evaluator errors on model outputs). Sections 3.3.1 and 5.1 and Appendix A.11 describe the same evaluator at different levels of detail: the main text summarizes it in one sentence, and the appendix specifies the value normalization. The revision unifies them into the following precise contract, which gives both the training reward and all evaluation scores. An output (the agent's final workbook for one test case) is accepted only if three checks pass on the oracle's answer region:
In the main-text summary, the “numeric tolerance” is the two-decimal rounding, the “formula-level check” is check 2, and structural edits are assessed through the answer-region values. Checks 2 and 3 make our evaluator stricter than SpreadsheetBench's official comparison, which scores values only. In the revision, Sections 3.3.1 and 5.1 and Appendix A.11 state this contract, and Appendix A.3 and the Limitations are aligned with it.
Hard-coded values. Check 2 rejects a plain value written where the oracle has a formula. Such outputs are rare and become rarer with RL: across the three new runs (Q3; new Appendix A.23), outputs that pass checks 1 and 3 but are rejected only for writing constants make up 0.06% of all SpreadsheetBench outputs before RL and 0.02% after.
Formatting-only requirements. Fill and font colors are scored in every task (check 3); other styles, such as bold or borders, are not. Tasks that would change only such unscored styles are removed during the construction of our training data and Domain-Spreadsheet, since their input already matches the target under this evaluator (Appendix A.1); SpreadsheetBench is used as released, so its scores stay comparable with prior work.
Evaluator errors on model outputs (Q1). In addition to the oracle audit in the paper, we audited model outputs (new Appendix A.23). From one new run (Q3), we randomly sampled 50 SpreadsheetBench outputs that the evaluator accepted and 50 that it rejected, at step 0 and at step 60. Three human auditors, blind to the checkpoint and to the evaluator's verdict, judged whether each output satisfies the instruction; unrequested edits outside the answer region count as violations.
Neither error rate increases after RL: false accepts fall from 7/50 to 2/50, with a 95% CI of the change that rules out an increase of more than 1.8 points, and false rejects stay at 2/50.
Correcting this run's measured Pass@1 (19.4% to 23.9%) with both audited rates (accepted share × (1 − false-accept rate) + rejected share × false-reject rate) gives 19.9% to 26.0%, so the audit gives no indication that evaluator errors inflate the gain.