acceptodds
Under review as a conference paper at ICLR 2027

FinEvolvingAgent: Self-Evolving Financial Deep Research Agent via Gradient-Guided Group Direct Preference Optimization

Abstract

Financial deep research represents one of the most demanding frontiers for autonomous AI agents, requiring complex multi-hop numerical reasoning, visual understanding of multimodal charts and financial tables, and strategic long-horizon tool invocation across regulatory filings. Existing reinforcement learning methods, such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), face severe bottlenecks in this domain due to critic instability, high rollout variance, and the absence of continuous preference guidance across complex multi-step reasoning trajectories. To overcome these limitations, we introduce , an autonomous multimodal financial deep research agent built upon the open foundation model Qwen3.6-VL. We first propose , which incorporates continuous differentiable reward signals into policy trajectory rollouts, directing exploratory token generation and tool actions toward mathematically consistent paths and achieving a +5.6% accuracy gain over base Qwen3.6-VL. Furthermore, we develop , a principled framework that generalizes direct preference alignment to multi-candidate group rollouts via a margin-aware Plackett-Luce objective, yielding an additional +3.2% improvement over GRPO. Across six extensive financial benchmarks, including FinQA, TAT-QA, ConvFinQA, FinanceBench, MultiFin-VL, and a newly constructed realistic multi-source benchmark RealFin-DeepResearch, FinEvolvingAgent achieves a new state-of-the-art accuracy of 83.8%, outperforming leading proprietary models including GPT-4o and Claude 3.5 Sonnet while exhibiting superior calibration, numerical precision, and grounded tool execution.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Self-Evolving Agent, Financial Deep Research

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_XKjQOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents FinEvolvingAgent, a system for multimodal financial analysis. It combines a composite reward for numerical accuracy, multimodal grounding, tool execution, and output structure with gradient-guided sampling, aiming to alleviate weak learning signals under sparse rewards. It also introduces a margin-aware Plackett–Luce group-ranking preference objective, termed G-GDPO. Experiments on six financial benchmarks, together with component ablations, report improvements in accuracy and training stability.

Critical strengths
  1. The reward decomposition is detailed and well aligned with practical financial analysis. Such tasks require accurate numerical extraction, consistent units and reporting periods, evidence localization, and reliable computation. Evaluating these capabilities separately provides a meaningful basis for training and assessing financial agents.

  2. Ranking groups of candidate trajectories is a reasonable way to exploit relative quality across multiple solutions and may provide richer supervision than a single binary comparison. The clear motivation also makes the intended role of this design easy to understand.

Critical weaknesses
  1. Potentially fabricated baseline references undermine confidence in the experiments. I could not locate papers matching the cited titles and authors for FinRobot 2.0 and Agent-R1; their arXiv IDs, 2601.08920 and 2602.04512, correspond to unrelated papers, W-DUALMINE and BrainVista. The relationship between MultiFin-VL and the cited FinReason-VL is also unexplained. These issues raise serious concerns about reference authenticity and the provenance of the baseline results in Tables 1 and 2.

  2. The core gradient-guidance mechanism is insufficiently specified for reproduction. Section 3.3 does not establish how Φ\Phi differentiably depends on ztz_t or specify its training objective and supervision. Appendix A.1 derives the gradient with respect to θ\theta, leaving the computation of ∇ztΦ(st,zt)\nabla_{z_t}\Phi(s_t,z_t) in Eq. (7) unresolved.

Questions for authors
  1. Please provide verifiable sources for the above baselines and benchmark, and explain the citation mismatches. How were the FinRobot 2.0 (70B) and Agent-R1 (72B) results in Tables 1 and 2 obtained? Please provide the model versions, implementations, evaluation protocols, and experimental records supporting these results.

  2. Please specify Φ\Phi's computational graph, training objective, supervision, and validation procedure, including how ∇ztΦ(st,zt)\nabla_{z_t}\Phi(s_t,z_t) is computed. Is guidance used at test time, and if so, with what inputs and computational overhead?

  3. DPO Appendix A.3 [1] already derives a Plackett–Luce preference objective. With Mj,k=0M_{j,k}=0, Eq. (9) matches that per-group loss with unit log-ratio scaling. What mathematical novelty does GDPO add, and what do adaptive margins and gradient-guided online sampling each contribute?

[1] Rafailov et al. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. NeurIPS, 2023.

Reviewer_cJcPOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes FinEvolvingAgent, a multimodal agent for financial deep research trained with two new policy optimization methods, G-GRPO and G-GDPO, and reports state-of-the-art results on six financial benchmarks.

However, several of its references appear to be hallucinated, including those for two of the baselines.

Critical strengths
  • The paper tackles a practical problem: answering financial questions that require reading filings and charts, calling tools, and getting exact numbers right. Giving partial credit to answers that are numerically close is a sensible choice for this setting.
Critical weaknesses
  • Several references appear to be hallucinated. The arXiv IDs given for FinRobot 2.0 and Agent-R1 point to unrelated papers, yet both are used as baselines in Tables 1 and 2. Other entries, such as Table-LLM and the step-level value preference optimization paper, give titles, authors, or venues that do not match the real papers. The comparison with these baselines therefore cannot be checked.
  • The method is not clearly explained, and its main ideas are not new. The paper never explains how the value network that guides the rollouts is trained, and the citation reward listed as a contribution does not appear in the reward definition. Ranking a group of responses with a Plackett-Luce model is already derived in the DPO paper, and steering generation with the gradients of an extra model is the core idea of PPLM, but the paper presents both as new without discussing this prior work.

Hallucinated references:

  1. FinRobot 2.0 (Yang et al., 2026; arXiv:2601.08920)
  2. Agent-R1 (Zhang et al., 2026; arXiv:2602.04512)
Questions for authors
  1. Could you provide correct references for FinRobot 2.0, Agent-R1, Table-LLM, and the step-level value preference optimization paper, and explain how the FinRobot 2.0 and Agent-R1 baselines were run?
  2. How is the value network that guides the rollouts trained?
Reviewer_b341OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper has serious problems with experimental reliability and reproducibility. The base model cannot be verified, key experimental results are inconsistent, some calculations are wrong, the financial examples do not match official reports, and several references are incorrect. These problems make the experimental results and conclusions hard to trust.

Critical strengths

The research problem is important. Financial deep research involves multi-step numerical reasoning, multimodal understanding, and tool use. Improving exploration efficiency and learning from sparse rewards is a useful research direction.

The main idea is reasonable. Combining continuous reward guidance with group-level preference ranking is worth exploring.

Critical weaknesses
  1. The base model used in all experiments does not exist.** All post-training baselines and ablation studies rely on Qwen3.6-VL-72B. However, this model does not exist. It is therefore unclear how these experiments were conducted, and the reported results cannot be trusted.

2. The claimed training scale is not supported by sufficient evidence. The authors claim to train a 72B model on 64 H100 GPUs with a 32K context length, 128 queries × 8 rollouts, and 4,000 optimization steps. This implies approximately 4.1 million trajectories for a single training run, an extremely expensive setup. More importantly, Figure 5 presents training curves for multiple baselines, with shaded bands reportedly representing standard deviations across five random seeds. The total computational cost of these experiments would be enormous. Yet the paper provides no training time, GPU-hours, rollout throughput, per-seed results, or numerical mean and standard deviation values. For experiments of this scale, a hyperparameter table and training curves alone are far from sufficient to substantiate the reported results.

3. Table 1 and Table 7 report conflicting results. In Table 7, RealFin-DeepResearch fails on only 51 out of 1,200 tasks, which would imply a success rate of 95.75%. However, Table 1 reports an accuracy of only 78.9%. Similar problems appear for SFT, PPO, and GRPO. The authors do not explain why these numbers are so different or how the two metrics are related.

4. The training curves do not match the reported results. In Figure 5, PPO ends at 74.8%, but Table 1 reports 77.2%. The improvement of G-GRPO is reported as 5.6 percentage points in the abstract and 6.4 percentage points in the main text. Figure 5 also claims that the final model improves over standard GRPO by 3.2 percentage points, while Table 1 shows a difference of 5.7 percentage points. These are basic numerical inconsistencies that should have been checked before submission. The paper does not clarify whether these results correspond to different training checkpoints or evaluation protocols.

5. The financial examples contain errors compared with official SEC filings. In the Microsoft FY2024 example, the authors report Intelligent Cloud operating income as $47,249 million, but the official 10-K reports $49,584 million. This error also affects the later calculations. The NVIDIA FY2025 example contains incorrect numbers for Data Center revenue and operating cash flow. The note number for segment information is also wrong.

6. Several references are wrong. The paper cites arXiv:2601.08920 as FinRobot 2.0, but this ID actually belongs to a medical image fusion paper. It also cites arXiv:2602.04512 as Agent-R1, but this ID belongs to BrainVista. The author information for FinanceBench is also incorrect. These errors include references to important baselines, which makes it difficult to verify the related

Questions for authors

As Above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.