- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents FinEvolvingAgent, a system for multimodal financial analysis. It combines a composite reward for numerical accuracy, multimodal grounding, tool execution, and output structure with gradient-guided sampling, aiming to alleviate weak learning signals under sparse rewards. It also introduces a margin-aware Plackett–Luce group-ranking preference objective, termed G-GDPO. Experiments on six financial benchmarks, together with component ablations, report improvements in accuracy and training stability.
Critical strengths
-
The reward decomposition is detailed and well aligned with practical financial analysis. Such tasks require accurate numerical extraction, consistent units and reporting periods, evidence localization, and reliable computation. Evaluating these capabilities separately provides a meaningful basis for training and assessing financial agents.
-
Ranking groups of candidate trajectories is a reasonable way to exploit relative quality across multiple solutions and may provide richer supervision than a single binary comparison. The clear motivation also makes the intended role of this design easy to understand.
Critical weaknesses
-
Potentially fabricated baseline references undermine confidence in the experiments. I could not locate papers matching the cited titles and authors for FinRobot 2.0 and Agent-R1; their arXiv IDs, 2601.08920 and 2602.04512, correspond to unrelated papers, W-DUALMINE and BrainVista. The relationship between MultiFin-VL and the cited FinReason-VL is also unexplained. These issues raise serious concerns about reference authenticity and the provenance of the baseline results in Tables 1 and 2.
-
The core gradient-guidance mechanism is insufficiently specified for reproduction. Section 3.3 does not establish how differentiably depends on or specify its training objective and supervision. Appendix A.1 derives the gradient with respect to , leaving the computation of in Eq. (7) unresolved.
Questions for authors
-
Please provide verifiable sources for the above baselines and benchmark, and explain the citation mismatches. How were the FinRobot 2.0 (70B) and Agent-R1 (72B) results in Tables 1 and 2 obtained? Please provide the model versions, implementations, evaluation protocols, and experimental records supporting these results.
-
Please specify 's computational graph, training objective, supervision, and validation procedure, including how is computed. Is guidance used at test time, and if so, with what inputs and computational overhead?
-
DPO Appendix A.3 [1] already derives a Plackett–Luce preference objective. With , Eq. (9) matches that per-group loss with unit log-ratio scaling. What mathematical novelty does GDPO add, and what do adaptive margins and gradient-guided online sampling each contribute?
[1] Rafailov et al. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. NeurIPS, 2023.