acceptodds
Desk rejected at ICLR 2027

Navalia: Advancing Cowork Agents through Verifiable Task Synthesis and Long-Horizon Post-Training

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model (LLM) agents are evolving from conversational assistants into Cowork Agents capable of autonomous planning, tool use, and complex task execution. Financial analysis requires these agents to integrate cross-document evidence and perform multistep numerical reasoning, but expert task authoring is costly, while direct LLM synthesis struggles to produce challenging tasks with reliable answers. We introduce Navalia, a framework for Cowork Agents that combines verifiable task synthesis with long-horizon post-training. Using corporate financial reports as its data source, Navalia semantically selects supporting evidence from complete fact sets and constructs executable computation chains to generate analytical tasks with traceable evidence and actual data dependencies between steps. We further use independent LLM review and programmatic checks to verify evidence consistency and answer correctness. The verified tasks are then used for trajectory collection, cold-start supervised fine-tuning (SFT), and reinforcement learning (RL) to improve long-horizon task execution. Navalia, initialized from Qwen3.6-35B-A3B, achieves consistent gains over the base model across all five benchmarks: OfficeQA Pro, FinSearchComp, Finance Agent Benchmark, APEX-Agents-AA, and Workspace-Bench-Lite. These results demonstrate that Navalia improves evidence integration, tool use, and long-horizon task execution capabilities and generalizes to broader Cowork settings, including general office tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Synthetic Data Generation, Agent Post-Training, Long-Horizon Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KxN8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces a framework, called Navalia, for training cowork agents which combines verifiable task synthesis with long horizon post training. The authors instantiate Navalia in the context of financial document analysis, showing gains over the base model on a suite of financial analysis benchmarks. For this framework, the authors generate syntheic data using a synthetic task generation pipeline, followed by long horizon reinforcement learning to train the specialize the model on these tasks.

Critical strengths
  • The authors explain the methods used in a clear and coherent manner.
Critical weaknesses
  1. The framework of synthetic task synthesis followed by reinforcement learning has been well established by many industry publications [DeepSeek-AI 2025, Chang 2026, Kabra 2026]. As a result, the novelty of the methods outlined in this work are unclear. I would hope for further clarifications for how this differs from prior work.

  2. The paper lacks any ablations, making it unclear what design decisions lead to the performance increase. It would be nice to provide some ablations of other training methods (both end to end and replacing parts of the pipeline) to see what is necessary.

  3. The paper should compare this framework to other possible frameworks for agentic training. At present, the only comparisons are to untrained large open and closed weight models.

[DeepSeek-AI 2025] DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models, https://arxiv.org/pdf/2512.02556

[Chang 2026] KARL: Knowledge Agents via Reinforcement Learning, https://arxiv.org/pdf/2603.05218

[Kabra 2026] Learning from Synthetic Data Improves Multi-hop Reasoning, https://arxiv.org/pdf/2603.02091

Questions for authors
  1. Can you clarify what the main novelty of this work is in relation to the aforementioned works? How does this differ from previously published agent training pipelines?
  2. Can you ablate data generation and RL pipelines? Specifically: is SFT necessary? How does this task synthesis compare to taxonomic synthesis as proposed by Deepseek?
  3. If I understand correctly, FIFO context management will hurt the KV cache during serving time? How would this compare to summarization?
3 comments
AuthorsOpenReview ↗

Response to Reviewer KxN8 (1/2): W1 / Q1

Thank you very much for your careful review.

W1 / Q1

Navalia's main distinction lies in how it constructs analytical tasks from real documents with traceable evidence, actual computational dependencies, and verifiable answers.

Compared with DeepSeek-V3.2, the main difference is the environment the agent operates in and how tasks are constructed for it. Its General Agent pipeline builds databases and task-specific tools around task categories, then generates tasks that require using these tools. Navalia instead retains a document environment composed of real financial reports, where the agent must locate evidence across files and tables, reconcile metric definitions and reporting scopes, and perform calculations. Our task construction therefore focuses on selecting evidence from existing documents that jointly supports an analysis, and designing computational tasks from that evidence. The internally extracted structured facts are used only for construction and verification, and are not provided to the solving agent.

Compared with KARL, the main difference is whether an explicit computation chain constrains question generation. KARL generates question-answer pairs from retrieved documents, then checks the samples through repeated solving and quality filtering. Navalia first specifies which facts to use, which operations to perform, and how each step uses the results of preceding steps. It computes the reference answer before generating the question from this chain. The requested operations and final answer are thus grounded in an explicitly recorded computation chain, allowing us to check evidence sources, computational results, and dependencies step by step, followed by independent review after generation.

Compared with Kabra et al., the main difference is the task setting. Their work uses rules to generate fictional facts and relations, and places all materials needed to solve a problem directly in the model's input. It focuses on training models to connect facts within those materials and perform multi-hop reasoning. Navalia instead constructs tasks from real documents and requires the agent to independently retrieve supporting evidence from a workspace containing distractor documents, check reporting periods, metric definitions, and units, and then perform multi-step numerical calculations. The internal fact database and computation chains used during construction are not provided to the solving agent.

We will add these comparisons to the main text and Related Work discussion to clarify the task differences and the corresponding designs for evidence selection, consistency checks, and computation-chain verification.

AuthorsOpenReview ↗

Response to Reviewer KxN8 (2/2): W2 / Q2, W3, and Q3

W2 / Q2

We use SFT as a cold start so that the model can first learn effective evidence retrieval, tool use, and multi-step analysis from high-quality teacher trajectories, providing a better initial policy for subsequent RL. For these long-horizon tasks, if the initial policy struggles to retrieve evidence and perform calculations effectively, rewards based on the final answer may provide insufficient learning signals.

We have added comparisons of training stages and sampling strategies. OfficeQA Pro accuracy is reported below:

SettingAccuracy
Initial model: Qwen3.6-35B-A3B39.60%
SFT: unbalanced sampling dominated by basic retrieval56.27%
SFT: balanced sampling59.94%
SFT + RL62.66%

SFT improves accuracy to 59.94%, and RL further raises it to 62.66%. Balanced sampling improves accuracy by 3.67 percentage points over sampling dominated by basic retrieval. These results support our training recipe of a cold start with high-quality trajectories, balanced task sampling, and subsequent RL.

In addition, Table 2 in the original manuscript compares FIFO with a per-turn sliding window to examine the role of context handling, while Figure 2(c) analyzes evidence retrieval before and after training. We will add the new ablation results and their experimental settings to the revised manuscript alongside these comparisons and analyses.

W3

We would like to clarify that models such as Kimi and GLM have already undergone targeted training for agentic capabilities. For example, Kimi's public report describes large-scale tool-use data synthesis and reinforcement learning, while GLM's report describes multi-step tool-call trajectory synthesis and reinforcement learning for search and coding agents. These training processes directly involve tool use, information retrieval, and multi-step reasoning. It is therefore inaccurate to characterize these comparison models as lacking training in the relevant capabilities.

Table 1 evaluates Navalia's overall performance relative to these strong models. The additional SFT, SFT + RL, and task-sampling ablations further examine the roles of individual training choices in Navalia. We will add the training background of the comparison models, along with the new experimental results and detailed settings, to the revised manuscript.

Q3

Removing early interactions under FIFO can reduce KV cache reuse. We update the window once every 10 new turns; unchanged prefixes can still be reused during the intervening turns.

We adopt FIFO primarily to control context length and keep the context consistent between training and trajectory generation. Compared with summarization, FIFO directly preserves recent interactions without requiring an additional summary-generation step. Summarization can retain important information from earlier interactions, but rewriting the history can also affect cache reuse. We will briefly clarify this trade-off in the revised manuscript.

I thank the authors for their response and am grateful for both the comparison and ablations that were done.

I agree with the authors that these open models have undergone significant agentic training. However not in a similar targeted manner. For example, I would have liked to see comparisons of Navalia's method for synthetic question generation to DeepseekV3 and KARL pipelines, using the same base model and corpus. I appreciate the discussion, but it otherwise is unclear to me the exact quality of the method. For this reason, I will maintain my score.

Reviewer_xJ87OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this manuscript, the authors present Navalia, a pipeline that synthesizes verifiable financial-analysis tasks from corporate financial reports and uses them to post-train a long-horizon agent. Tasks are built from a dual document/fact representation: an LLM selects supporting facts, executable computation chains with explicit data dependencies produce the reference answers, and independent LLM review plus programmatic checks filter the results. The verified tasks are then used for cold-start SFT on teacher trajectories and for GRPO-style RL, in which trajectory-level advantages are shared across FIFO context segments. Starting from Qwen3.6-35B-A3B, Navalia improves over the base model on all five benchmarks (+13 to +23 points), including two general office benchmarks outside finance. Overall, this is a practically useful engineering contribution with strong headline gains, but the paper currently provides little evidence about which components produce them.

Critical strengths

Strengths

  1. The task synthesis design, which enforces real data dependencies between steps and executes complex computations deterministically, is a sensible response to known weaknesses of LLM-generated QA (shortcut lookups, arithmetic errors), and every task remains traceable to its source evidence.

  2. Gains over the base model are consistent across five benchmarks, including transfer to APEX-Agents-AA and Workspace-Bench-Lite under different harnesses, which suggests the learned skills are not narrowly finance-specific.

  3. The end-to-end recipe (trajectory filtering, segment-level loss masking, handling of truncated rollouts, task selection for RL) is described concretely, and the planned release of weights and pipeline components would be valuable given how little of this is publicly documented.

Critical weaknesses

Major issues

  1. There are no ablations isolating the contributions. The paper reports only the base and final models: there is no SFT-only versus SFT+RL comparison, no comparison with tasks synthesized directly by an LLM (without semantic fact selection, computation chains or verification), and no test of advantage sharing across segments versus training on the anchor segment alone. Without these, the central claim that verifiable task synthesis drives the gains cannot be separated from generic distillation from a strong teacher, whose identity is not stated. The paper also gives no statistics for the synthesis pipeline itself (number of tasks generated, pass rates at each verification stage, a human audit of reference-answer correctness). The FIFO comparison in Table 2 uses different numbers of trajectories (6,826 vs 6,031) and a 1.6-point difference on 133 questions from a single run, which is within noise.

  2. Evaluation rigor and possible contamination need attention. OfficeQA Pro and the Finance Agent Benchmark draw on public financial documents that may overlap with the training corpus; the authors should report document-level overlap between the training environments and the benchmark corpora. All results are single runs, and OfficeQA Pro (133 questions, standard error of roughly 4 points) and Finance Agent (77 questions) are small, so means and standard deviations over repeated runs are needed. Many comparison numbers are taken from other reports or obtained with different harnesses (marked * and †), so claims of parity with frontier models are not controlled comparisons.

Minor issues

  1. The text and Table 1 are inconsistent. Section 4.1 lists a set of comparison models that differs from the table, which additionally includes Kimi-K3, Qwen3.5-397B-A17B, Claude Opus 4.7 and Seed2.1 Pro without description. Section 4.2 emphasizes outperforming Kimi-K2.6 but does not mention that Kimi-K3, DeepSeek-V4 Flash, Qwen3.8 Flash Next and GLM-5.2 exceed Navalia on most benchmarks, and the 3.02-point gain over Qwen3.7-Plus on APEX-Agents-AA is against a reported (*) value. Appendix C is cited for scoring protocols but is not included, and Equations 2, 5 and 8 contain rendering errors.

  2. Several method details are missing: the teacher model and the LLMs used for fact selection and review (and whether they share a model family, which risks correlated errors), the number of RL steps, group size G, clipping values, the numerical tolerance used to score answers, and the compute budget. The axes and labels of Figure 2 are not legible.

Questions for authors

N/A

1 comment
AuthorsOpenReview ↗

Response to Reviewer xJ87

Thank you for your careful review and your specific suggestions on the experiments, evaluation settings, and methodological details. We respond point by point below.

Major issue 1: Ablations and the roles of individual designs

We have added ablations of training stages and task sampling. All results below are OfficeQA Pro accuracy; the sampling comparison is conducted at the SFT stage.

AblationBaseline settingWith the componentGain (percentage points)
Reinforcement learningSFT only: 59.94%SFT + RL: 62.66%2.72
Balanced samplingUnbalanced sampling dominated by basic retrieval: 56.27%Balanced sampling: 59.94%3.67

These results show that RL provides further gains beyond SFT on teacher trajectories. Task distribution also matters: preventing basic retrieval tasks from dominating the data improves downstream performance. These two experiments examine the roles of subsequent RL and task sampling, respectively. We will include their results and detailed settings in the revised manuscript.

Regarding task synthesis, Section 2.5 already includes two types of verification: an independent review model solves each question again from the question and supporting facts, without reusing the reasoning from generation; programmatic checks verify computational dependencies and reporting-period consistency.

We will also report synthesis-pipeline statistics, including the numbers of generated and retained tasks and the pass rate at each verification stage, together with the sample size, procedure, and results of a manual audit, to provide more direct evidence of data reliability.

The accuracies in Table 2 are averages over six repeated evaluations (avg@6), rather than single-run results. This experiment primarily examines sample expansion under different trajectory segmentation methods. FIFO reduces the average number of samples per trajectory from 40.52 to 3.90, a 90.4% reduction, while maintaining comparable downstream performance. This is precisely the point emphasized in Section 4.3: reducing training-sample expansion caused by repeated context while maintaining task performance.

Major issue 2: Repeated evaluations, data overlap, and comparison settings

The main-table OfficeQA Pro results were evaluated six times, and Finance Agent Benchmark nine times, with the latter reported as the average (avg@9). Thus, neither benchmark's results are based on a single run. The original manuscript did not explicitly state the repetition counts; we will add these and the aggregation method to Section 4.1 and the table notes.

Regarding data overlap, we checked the evaluation questions and associated environment files of both OfficeQA Pro and Finance Agent Benchmark against Navalia's training tasks and environment files, and found no question- or document-level overlap. Although both training and evaluation involve financial document analysis, neither benchmark's evaluation questions or documents were used to construct training tasks.

Navalia, the initial model, and most open-weight baselines were evaluated by us using the same harness, system prompt, tool interfaces, and interaction budget within each benchmark. Thus, Navalia's gains over the initial model and comparisons with these open-weight models are based on consistent evaluation settings. Closed-source results are mainly taken from public reports as external references. Table 1 already marks reported results and those obtained with different document representations or harnesses.

Minor issue 1: Consistency between the text and tables, and formatting

Our goal is to provide a complete training recipe for improving Cowork Agent capabilities, with gains over the initial model verified across multiple benchmarks. Navalia approaches frontier-model performance on some metrics, although an overall gap remains under our current training-data scale and compute resources. We will discuss these gaps in Section 4.2 to present a fuller account of Navalia's performance, and add the omitted baseline descriptions to Section 4.1.

Appendix C is already included on page 2 of the submitted supplementary material (manuscript page 13), where it describes the evaluation settings and scoring protocols for each benchmark. We will also fix the rendering errors in Equations 2, 5, and 8.

Minor issue 2: Methodological details and figure readability

Appendix B of the supplementary material already provides the SFT epoch count, batch size, sequence length, and trajectory-collection settings. We will provide further information on the teacher and auxiliary models and other training parameters to the extent that it can be publicly disclosed, and improve the readability of the axes and labels in Figure 2.

Reviewer_mA3uOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Navalia, a framework combining verifiable task synthesis with SFT and RL to improve long-horizon agent capabilities. Experiments on five benchmarks demonstrate consistent improvements over the base model.

Critical strengths
  1. The paper proposes a task synthesis pipeline that generates analytical tasks with traceable evidence, explicit computational dependencies, and verifiable answers.
  2. The paper presents a post-training pipeline combining trajectory collection, SFT, and long-horizon RL.
Critical weaknesses
  1. Insufficient methodological novelty and differentiation from prior work. The framework mainly integrates existing techniques for task synthesis and post-training, without clearly distinguishing itself from related works such as WebShaper and ClawGym. The lack of a systematic Related Work discussion further weakens its novelty claims.
  2. Insufficient experimental validation of the claimed contributions. The paper lacks comparisons with existing task synthesis methods, direct evaluations of synthesized data quality, and ablation studies to justify the effectiveness of its key components.
  3. Unclear presentation of the proposed methodology. Figure 1 is difficult to follow, and several methodological details are insufficiently explained. A concrete example illustrating the task synthesis process would improve clarity.
Questions for authors

Please see the concerns raised in the Weaknesses section, particularly regarding methodological novelty and insufficient experimental validation.

1 comment

Thank you very much for your careful review.

W1

We would like to clarify how Navalia differs from these two works.

Compared with WebShaper, the main difference is the task setting. WebShaper focuses on multi-hop web search QA, whereas Navalia targets cross-document retrieval and multi-step numerical analysis over real documents, combining evidence localization, alignment of metric definitions and reporting scopes, and computation.

Compared with ClawGym, the main difference is how tasks are constructed. Both address Cowork-style tasks, but ClawGym first generates tasks and then creates matching mock files. Navalia starts by collecting, downloading, and parsing real documents, and constructs tasks from existing evidence. This requires checking evidence coverage, aligning reporting periods and metric definitions, and constructing and verifying computation chains with actual dependencies between steps.

We did not include these two works as experimental baselines because WebShaper primarily targets search tasks, while the original ClawGym paper does not report results on the benchmarks we use. We will clarify these differences and the corresponding methodological designs in the main text and Related Work discussion.

W2

We have added two ablation experiments and would also like to clarify the data verification and experimental analyses already included in the manuscript.

Regarding data reliability, Section 2.5 uses an independent review model to solve each question again from the question and supporting evidence, without reusing the reasoning from the generation stage. Programmatic checks then verify computational dependencies and reporting-period consistency. Only tasks that pass both checks are retained.

For the key training choices, the new ablation results are shown below. All scores are OfficeQA Pro accuracy, and the balanced-sampling comparison is conducted at the SFT stage.

AblationBaseline setting and resultWith the componentGain (percentage points)
Reinforcement learningSFT only: 59.94%SFT + RL: 62.66%2.72
Balanced samplingUnbalanced sampling dominated by basic retrieval: 56.27%Balanced sampling: 59.94%3.67

These results show that RL further improves performance after SFT. Balanced sampling is also beneficial: preventing basic retrieval tasks from dominating the training data leads to better downstream performance.

Table 2 in the original manuscript also compares FIFO with a per-turn sliding window. With both settings allowing at most 30 visible interaction turns, the average number of samples per trajectory decreases from 40.52 to 3.90, a 90.4% reduction, while OfficeQA Pro accuracy increases from 49.96% to 51.56%. This supports FIFO's ability to reduce sample expansion caused by repeated context while maintaining downstream performance.

Furthermore, Figure 2(c) shows that, after training, the proportion of tasks for which all supporting documents are retrieved increases from 45.4% to 69.8%, and document recall rises from 50.8% to 74.7%. Thus, improved task performance is accompanied by better evidence retrieval, with the model obtaining a more complete set of documents needed to solve the task.

We hope these results more directly address your concern about the effectiveness of the key training choices. We will add the new ablation results and their experimental settings to Section 4.3 of the revised manuscript to present the evidence for these choices more fully.

W3

We will improve the connections between modules and the input/output labels in Figure 1, and add a concrete example.

Using illustrative numbers, we select revenues of 10, 12, and 14 billion yuan for three consecutive years from financial reports, ensuring consistent metric definitions and reporting scopes. A linear trend fit predicts revenue of 16 billion yuan for the following year. We then calculate the growth rate relative to the most recent year, obtaining 14.29%. The question is generated from this computation chain, and independent review and programmatic checks verify the answer and the dependencies between steps.

We will map this example to each stage in Figure 1 to make evidence selection, computation-chain construction, and verification easier to follow.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.