- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies tool selection for a frozen LLM reasoner on multiple-choice genomics questions. A policy (Qwen2.5-7B with LoRA) chooses which of four sequence-analysis tools to run before the reasoner answers: composition, motif scan, splice-site scoring, and a kNN predictor over Nucleotide Transformer embeddings. The authors argue that GRPO is ill-suited when all 2⁴ = 16 tool subsets can be enumerated. As the policy concentrates, the sampled rollouts often all get the same reward, so the group-normalized advantage is zero (a "dead group"). The closed-form dead-group rate on GenomeQA rises from 0.2% under a uniform policy to 20.8% after GRPO training.
The authors propose FGPO: precompute the reasoner's reward for every question–subset pair once, then maximize the exact expected reward over all 16 subsets. The objective uses a length-normalized softmax over the candidate strings plus an entropy bonus. Across three benchmarks (GenomeQA plus two suites, GenBench-X and BM4) and five frozen reasoners, FGPO beats GRPO in all 15 settings, by 6.75 points on average. It also calls fewer tools (1.40 vs 2.36 per question on GenomeQA). The paper also covers cover: (a) a sweep over how many of the 16 subsets the objective sees per question; (b) an ablation of the training objective (cross-entropy, regression); (c) checkpoint and hyperparameter sweeps for the baselines.
Critical strengths
- Careful diagnosis and many controls. The paper gives a closed-form dead-group analysis and varies how many subsets the objective sees, with accuracy rising as it sees more. All arms are also scored under the same inference rule, the objective is ablated, the baselines get checkpoint sweeps, and differences are tested with question-paired McNemar tests. This makes the paper's own measurements easy to audit.
- Consistent margins over GRPO with fewer tool calls. FGPO beats GRPO in all 15 benchmark–reasoner settings, including four reasoners the policy was never trained with, while using fewer tools and 3.5–4.1× fewer input tokens than calling all tools.
Critical weaknesses
-
The learned routing adds little over a fixed choice, and the contribution is largely known. FGPO calls the kNN tool (T4) on 86–96% of questions and never calls T1. App. C.3 states that FGPO only "tracks" the best fixed subset, chosen using test labels, with a gap that goes both ways, and that baseline isn't in Table 2. A dev-selected "always use the best fixed subset" baseline, e.g., always T2+T4 or always T4, would likely match FGPO. The method claims a query-dependent policy, so this undercuts the main claim.
Methodologically, maximizing the exact expected reward over a small known action set with a full reward table is a full-information contextual bandit, close to cost-sensitive classification and to expected policy gradients (Ciosek & Whiteson). A plain regression on the reward table ties FGPO on BM4. That GRPO underperforms in this regime is expected.
-
The observations from baselines and benchmarks are not convincing.
- Every baseline that could compete collapses to the no-tool floor at every checkpoint: DPO, GRPO trained on Eq. 1, and DAPO trained on Eq. 1, all within 0.4 points of No-Tool. Yet FGPO, on the same reward, calls T4 on 86% of questions. DAPO's dynamic sampling exists precisely to discard zero-advantage groups, so its collapse points to an implementation or reward problem rather than an inherent limit of sampling.
- The reported GRPO uses a different reward from FGPO, and it scores below simply calling all tools.
- GenBench-X and BM4 are not established benchmarks. The authors built them from Genomic Benchmarks, DeepSTARR and other sources. Label meanings were inferred heuristically, and two turned out reversed. Sources were assigned to task slots by hand, e.g., DeepSTARR enhancers filed as "TF motif".
- T4 is a kNN classifier built on labelled training splits of the same sources, so "tool selection" largely reduces to "consult the classifier".
- There is one training run per arm, with no seed variance.
- The "2.4× fewer reasoner calls" figure is bookkeeping, not a measured saving: a cached GRPO needs at most the same 32,032 unique calls.
Questions for authors
- Most task families here (promoter/enhancer, splice site, histone marks, taxonomy) are standard sequence-classification tasks. On these, fine-tuned genomic models (e.g., DNABERT-2, Nucleotide Transformer, HyenaDNA, Evo) are the usual baselines, and the multiple-choice format can be handled by scoring each option's sequence. How does the FGPO pipeline compare with (a) the T4 kNN classifier used directly as the answer, with no LLM, and (b) a fine-tuned genomic classifier on the same splits? If these match or exceed the pipeline, what does an LLM with learned tool selection add for these tasks?
- How does FGPO compare in Table 2 with a dev-selected best fixed subset and with "always T4", on all 15 settings?
- Why does DAPO, whose dynamic sampling removes zero-advantage groups, still collapse to no tools at every checkpoint? Did you try GRPO with G=16, sampling without replacement, or the same entropy bonus FGPO uses?
- How were GenBench-X and BM4 built and labelled, how were sources assigned to task slots, and how was label correctness verified?
- Can you report results over several training seeds for FGPO and GRPO?