acceptodds
Under review as a conference paper at ICLR 2027

ToolSciVer: Multimodal Scientific Claim Verification with Visual Tool Augmented Reinforcement Learning

Abstract

Multimodal scientific claim verification (MSCV) checks claims against text, tables, and figures of scientific papers. It requires extracting multiple pieces of localized evidence and combining them through multi-step reasoning; vision-language models (VLMs) struggle with both. Generic crop-and-zoom tools only return pixels, and prompting VLMs to use tools can even hurt accuracy. We present \ours, which teaches VLMs when and how to acquire claim-relevant evidence. It pairs SciEviKit, a type-aware tool suite returning structured evidence (table rows or columns, chart values, zoomed regions), with two-stage training: On-Policy Self-Distillation (OPSD) bootstraps tool use from reference tool calls alone, and Group Relative Policy Optimization (GRPO) optimizes a reward coupling accuracy with tool-call validity and efficiency. We evaluate on SciVer and MuSciClaims, spanning diverse evidence modalities, domains, and reasoning structures, with three backbones (Qwen, InternVL, Gemma) and five baselines covering non-tool reasoning, training-free agents, and RL-based tool use. \ours improves over non-tool chain-of-thought by 8.16 points on SciVer and 7.92 on MuSciClaims, and over the strongest baselines by 3.70 and 2.23 points. Our analyses further show better evidence acquisition, tool selection, and evidence-based reasoning, and ablations confirm the value of OPSD warm-up.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
VLM, RL, tool-use, claim verification

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_eDv6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents ToolSciVer, a framework for multimodal scientific claim verification. It combines SciEviKit, a toolkit for extracting table rows/columns, parsing charts, and inspecting image regions, with a two-stage training procedure: on-policy self-distillation using reference tool calls as privileged information, followed by GRPO with rewards for answer correctness, formatting, tool-call validity, and efficiency. Evaluation covers SciVer and MuSciClaims using three approximately 4B-scale VLM backbones. The paper reports average improvements of 8.16 and 7.92 percentage points over non-tool CoT, respectively. Additional experiments examine warm-up strategies, tool efficiency, evidence acquisition, verification given gold evidence, and transfer to external chart and figure benchmarks.

Critical strengths
  1. A useful system design with improvements across multiple backbones and benchmarks. SciEviKit addresses concrete difficulties in scientific verification: extracting precise table entries, recovering chart content, and inspecting localized figure evidence. Table 2 reports higher overall accuracy than the compared existing methods for all three backbones on both datasets. Table 12 also shows that OPSD warm-up improves over tool-enabled GRPO without warm-up across these settings. This breadth makes the proposed combination promising beyond a single backbone or benchmark.

  2. The evaluation distinguishes evidence acquisition from reasoning with evidence. Tables 5 and 6 provide complementary diagnostics: REAR increases from 83.00% to 90.00%, while verification accuracy with identical gold evidence increases from 60.33% to 71.67%. The latter includes a paired significance test. The external chart/figure evaluation also reports improvements without tool-use instructions. These analyses strengthen the empirical contribution by examining more than final task accuracy alone.

Critical weaknesses
  1. W1 — The reported evaluation does not establish an independent model-selection protocol. Section 4 and Appendix B.3 state that checkpoints are evaluated every 50 GRPO steps and that the checkpoint with the highest average evaluation accuracy is selected. Appendix B.1 specifies training and final evaluation/test sets but does not identify a separate validation set. Consequently, it is unclear whether the reported evaluation data also influenced checkpoint selection. If so, selecting among roughly 30 checkpoints could introduce optimistic bias into the reported gains. Moreover, the three reported runs vary only inference seeds for a fixed trained checkpoint; they do not establish robustness across training runs. This matters particularly for relatively small margins, such as the 0.71-point improvement over the strongest listed baseline on MuSciClaims with InternVL. An explicit split definition and results from checkpoints selected independently of the test data are necessary to resolve the central concern.

  2. W2 — The additional value of the proposed policy is not sufficiently separated from the stronger evidence-extraction backend. Appendix A.1 states that Chart Parse uses GPT-6-Astra with maximum reasoning effort, while Appendix B.4 states that VTool-R1 and OpenThinkIMG retain their original tool suites. Their comparison therefore changes both the learning procedure and the evidence available through tools. The results demonstrate the performance of the combined system, but do not cleanly establish the advantage of the proposed policy over alternatives with equivalent extraction capabilities. The shared-tool ablations in Table 3 partially address this issue and support a warm-up benefit. However, a task-trained verifier supplied with the same pre-extracted evidence would be a particularly informative control for establishing the value of adaptive tool acquisition. Likewise, fewer verifier tokens and tool calls in Table 4 do not establish lower end-to-end cost when a call can invoke a much stronger model. These gaps limit the demonstrated significance and practical efficiency of the proposed design.

Questions for authors
  1. Clarification concerning W1: Which exact split was used for checkpoint selection? Was it disjoint from the 1,000 SciVer and 505 MuSciClaims examples used for the reported results? If those examples influenced selection, can you provide evaluation on untouched data using a selection rule fixed beforehand?

  2. Additional evidence concerning W2: How does ToolSciVer compare with a task-trained verifier that receives the same OCR and chart-parser outputs directly, without learning when to request them? Alternatively, do the improvements over trained tool-use baselines persist when they receive the same SciEviKit backend and interaction budget?

  3. Clarification concerning W2: Are chart-parser outputs precomputed or generated during each interaction, and what backend computation is included in the efficiency measurements? Please distinguish verifier-token savings from total system cost, including extraction or preprocessing.

7 comments
AuthorsOpenReview ↗

Response to Reviewer eDv6(1/3)

Dear Reviewer eDv6,

Thank you for your thoughtful comments and constructive suggestions! We address your concerns below.


W1/Q1: Checkpoint Selection & Multi-Seed Result

Thanks for your suggestions! To address your concern on checkpoint selection, we 1) clarify the fairness of the comparisons in the original experiment, and 2) provide new results by selecting the best checkpoint on a disjoint evaluation set and reporting its performance on the test set.

First, the same checkpoint-selection and reporting protocol was applied to ToolSciVer and all training-based baselines, including VTool-R1 and OpenThinkIMG. The reported comparisons therefore use a consistent selection criterion across these methods. Similar reporting practices also appear in prior work. For example, OPSD[1] reports the best score across periodically evaluated checkpoints and reports peak performance for its GRPO baseline. We nevertheless acknowledge that a consistent protocol and prior precedent alone do not address the concern about separating checkpoint selection from final reporting.

To further address your concern about this, we partitioned each original test set into disjoint evaluation and test subsets in a 1:2 ratio, selected the best checkpoints solely based on evaluation set performance, and reported their accuracy on the test set. Under this protocol, ToolSciVer achieves the highest mean overall accuracy in all six dataset–backbone combinations. On SciVer, it outperforms the strongest baseline by 4.35, 2.19, and 3.80 percentage points for Qwen3.5-4B, InternVL3.5-4B, and Gemma4-E4B, respectively. On MuSciClaims, the corresponding improvements are 0.20, 0.99, and 1.28 percentage points. Although the margins vary across settings, ToolSciVer consistently ranks first, supporting the robustness of our comparative conclusions under evaluation-based checkpoint selection.

BackboneMethodSciVer (%)MuSciClaims (%)
Qwen3.5-4BCoT w/o SciEviKit72.16 ± 7.1868.94 ± 3.12
Qwen3.5-4BChartAgent78.46 ± 0.8381.11 ± 0.86
Qwen3.5-4BDocLens61.56 ± 0.6274.98 ± 1.04
Qwen3.5-4BVTool-R180.41 ± 0.4876.85 ± 0.51
Qwen3.5-4BOpenThinkIMG76.11 ± 0.6275.17 ± 1.12
Qwen3.5-4BToolSciVer84.76 ± 1.5481.31 ± 1.07
InternVL3.5-4BCoT w/o SciEviKit65.74 ± 0.3270.62 ± 4.78
InternVL3.5-4BChartAgent54.92 ± 4.3565.68 ± 2.82
InternVL3.5-4BDocLens64.37 ± 0.3570.72 ± 2.11
InternVL3.5-4BVTool-R168.77 ± 1.1674.98 ± 0.86
InternVL3.5-4BOpenThinkIMG67.22 ± 0.6176.26 ± 0.89
InternVL3.5-4BToolSciVer70.96 ± 2.3977.25 ± 2.52
Gemma4-E4BCoT w/o SciEviKit77.11 ± 1.6673.59 ± 0.51
Gemma4-E4BChartAgent74.61 ± 1.6574.98 ± 1.49
Gemma4-E4BDocLens72.71 ± 1.6274.68 ± 2.15
Gemma4-E4BVTool-R175.31 ± 1.6270.72 ± 1.20
Gemma4-E4BOpenThinkIMG74.41 ± 1.4272.80 ± 2.08
Gemma4-E4BToolSciVer80.91 ± 1.1176.26 ± 1.57

We additionally conducted multi-seed training experiments with Qwen3.5-4B for ToolSciVer, VTool-R1, and OpenThinkIMG, reporting mean accuracy and standard deviation across training runs. ToolSciVer achieves the highest mean accuracy on both benchmarks, outperforming the strongest baseline, VTool-R1, by 3.80 and 2.77 percentage points on SciVer and MuSciClaims. These results support the robustness of ToolSciVer’s performance gains across training seeds.

MethodSciVer (%)MuSciClaims (%)
VTool-R180.97 ± 2.7477.03 ± 1.58
OpenThinkIMG75.43 ± 3.0276.50 ± 2.00
ToolSciVer84.77 ± 2.0679.80 ± 0.71

References:

[1] Zhao, S., et al. 2026. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. ICML 2026.

AuthorsOpenReview ↗

Response to Reviewer eDv6(2/3)

W2/Q2: Contributions of Policy Learning and Evidence Extraction

Thank you for suggesting these comparisons. Our additional experiments show that ToolSciVer’s benefits extend beyond access to a stronger extraction backend, highlighting the importance of its learned tool-use policy. Specifically, three complementary comparisons show that (1) ToolSciVer outperforms a direct-evidence verifier on MuSciClaims while achieving comparable accuracy on SciVer; (2) its advantage over VTool-R1 and OpenThinkIMG persists when these baselines receive the same SciEviKit backend and interaction budget; and (3) it maintains strong performance when its own backbone replaces the stronger chart parser. We present these three comparisons below.

1. Directly providing pre-extracted evidence.

To assess the value of adaptive evidence acquisition, we trained a Qwen3.5-4B direct-evidence verifier that receives all available precomputed OCR and chart-parser outputs directly in its initial prompt, alongside the original images, without learning to request tools.

MethodSciVer Acc. (%)MuSciClaims Acc. (%)
Direct-evidence verifier85.03 ± 0.3577.43 ± 1.39
ToolSciVer84.07 ± 1.2780.79 ± 0.52

The direct-evidence verifier achieves 0.96 percentage points higher accuracy on SciVer, where tables and charts provide structured textual evidence that can be supplied upfront. MuSciClaims, however, consists of multi-panel scientific figures, for which this setup supplies no additional OCR or chart text. Inspecting relevant regions through zoom-in requires selecting where to look, which the direct-evidence verifier does not perform. But ToolSciVer’s learning policy supports this adaptive region inspection and achieves 3.36 percentage points higher accuracy on MuSciClaims.

These results highlight the value of adaptive evidence acquisition when verification requires inspecting claim-relevant image regions. Our goal is to learn whether additional evidence is needed, which visual tool to invoke, and which region to inspect, extending verification beyond evidence supplied upfront.

2. Equipping training-based baselines with SciEviKit.

To examine whether ToolSciVer’s advantage persists when baselines receive the same tools, we conducted a toolset replacement experiment on Gemma4-E4B. We retained the trained VTool-R1 and OpenThinkIMG checkpoints, replaced their original toolsets with SciEviKit, and updated the corresponding tool descriptions in their prompts. We then evaluated both methods on SciVer and MuSciClaims using the same SciEviKit backend and inference-time interaction budget as ToolSciVer. The results are in the following table.

MethodToolsetSciVer (%)MuSciClaims (%)
VTool-R1Original74.23 ± 1.1668.78 ± 0.80
VTool-R1SciEviKit77.20 ± 1.0071.16 ± 1.65
OpenThinkIMGOriginal72.40 ± 1.6171.16 ± 0.98
OpenThinkIMGSciEviKit77.63 ± 0.5870.76 ± 0.70
ToolSciVerSciEviKit81.20 ± 0.9574.65 ± 0.59

With SciEviKit, VTool-R1 scores 2.97 and 2.38 percentage points higher than its originally reported results on SciVer and MuSciClaims. OpenThinkIMG also scores 5.23 points higher on SciVer. ToolSciVer remains ahead of both baselines when they are equipped with SciEviKit, with margins of 3.57 and 3.49 points over the stronger baseline on the two benchmarks.

Thus, ToolSciVer’s advantage persists when the extraction backend and interaction budget are shared, supporting the contribution of its learned tool-use policy beyond access to SciEviKit.

3. Chart Parser with Qwen3.5-4B.

Finally, to assess whether ToolSciVer’s gains depend on a stronger external chart parser, we replaced Astra with Qwen3.5-4B itself and retrained and evaluated ToolSciVer using the resulting self-generated chart parses.

SettingSciVerMuSciClaimsChart-type problem
ToolSciVer (Astra parser)84.07 ± 1.2780.79 ± 0.5283.31 ± 0.33
ToolSciVer (self-parsing)83.10 ± 0.6279.21 ± 0.9180.59 ± 0.31
VTool-R181.18 ± 0.0376.78 ± 0.7578.33 ± 0.45
OpenThinkIMG75.94 ± 0.2375.60 ± 0.9472.92 ± 0.65

Compared with Astra-parsing, self-parsing reduces overall accuracy by 0.97 and 1.58 percentage points on SciVer and MuSciClaims, with a larger decrease of 2.72 points on chart-type problems. Nevertheless, it still outperforms both tool-use baselines, exceeding VTool-R1 by 1.92 and 2.43 percentage points on the two benchmarks.

Thus, ToolSciVer retains its advantage over these tool-use baselines when both training and evaluation use self-generated chart parses, demonstrating that its gains do not require a stronger external chart parser.

AuthorsOpenReview ↗

Response to Reviewer eDv6(3/3)

W2/Q3: Tool Execution and Efficiency

Chart-parser outputs are precomputed and retrieved on demand when the verifier invokes the tool. In ToolSciVer, Chart Parse accounts for 44.80% of all tool-call attempts, while the average number of tool-calls per example is 0.74(Table 4).

To further validate the effectiveness of seffs_{\mathrm{eff}} on token efficiency, we extended our token-efficiency analysis. 1) First, all of our experiments(also including these two variants) use the same precomputed chart-parsing and OCR results, so offline preprocessing is a shared, one-time cost(615k tokens in total, amortized 409 tokens per example). These outputs are cached and reused across subsequent experiments on the same data, without repeating preprocessing. We therefore report this cost separately and exclude it from our comparison of verification-stage token efficiency. 2) We then calculated the average number of tool-returned text tokens per example for ToolSciVer, obtaining 843 tokens. Adding these to the tokens generated by the verifier itself yields 1362 + 843 = 2205 text tokens per example. This remains 20.11% lower than the 2760 verifier-generated tokens of the variant without seffs_{\mathrm{eff}}, even before including that variant’s tool-returned text. Thus, the token-efficiency advantage persists when tool-returned text is included.

We also compare ToolSciVer’s 1) tool-call efficiency and 2) token efficiency with other baseline methods: VTool-R1 and OpenThinkIMG. The following table shows the tool-call efficiency; the metric in the table is average tool-call number per example.

MethodSciVerMuSciClaimOverall
VTool-R11.811.841.82
OpenThinkIMG1.931.741.87
ToolSciVer w/o seffs_{\mathrm{eff}}1.240.661.05
ToolSciVer w/ seffs_{\mathrm{eff}}0.850.540.74

From the table, we can see ToolSciVer use less tool-calls while achieves better performance(Table 2). Showing ToolSciVer’s good efficiency. The average text-token usage per example, including both reasoning tokens and tool-result tokens but excluding the shared, one-time preprocessing cost discussed above, is 2474 for VTool-R1 and 2324 for OpenThinkIMG, both higher than ToolSciVer’s 2205 tokens. These comparisons show that ToolSciVer achieves higher verification accuracy with fewer tool calls and lower text-token usage, highlighting its advantages in both tool-call efficiency and token efficiency.

Reviewer_BMZ9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a new method on training a 4B VLM model for multimodal scientific claim verification using a type-aware tool suite, SciEviKit. The tool kit actually involves 3 tools, OCR to csv-like table, chart parsing with a stronger VLM that converts charts to structured text, and region zoom that crop and enlarge a region. The training invovles 2 stages, the first stage of OPSD tries to make the small VLM learns how to call these tools, and the second stage use GRPO to teach how to let the VLM to make use of the tools to solve downstream tasks. On SciVer and MuSciClaims, ToolSciVer improves over non-tool chain-of-thought and over the strongest baselines. Further analyses cover tool routing, evidence acquisition, reasoning with gold evidence, and transfer to chart QA benchmarks.

Critical strengths
  1. using type-aware tools to return the strucutred evidence, but not directly using pixels is a reasonable design.
  2. the OPSD to distill reference tool-calls, without full reasoning demonstration is a reasonable warm-up strategy.
  3. the evaluation overall is broad, covering diverse set of benchmark and datasets.
Critical weaknesses
  1. The chart reading is delegated to a stronger frontier model. from my perspective, I think the chart reading itself is a core challenges in the claim verification, and delegating it to a stronger GPT model may not that reasonable. I think this weakens the claim that ToolSciVer lets small VLM perform comparible to the close source frontier models. The comparison with the external baselines may also potentially be unfair given they are using their original tools.

  2. It seems that the checkpoints are selected on the test set according to the descriptions in the paper.

  3. The benefit of the proposed component of OPSD is not that clear. GRPO without any tools already recovers most of the improvements, and it seems that in-domain RL is driving most gains. OPSD adds little over no warm-up, although it is a well-motivated thing to do.

Questions for authors
  1. How does performance change if Chart Parse uses an open model, and what fraction of tool calls go to Chart Parse?
  2. Is checkpoint selection performed on the test sets? If so, can you report results with selection on a held-out validation split and with multiple training seeds?
  3. How do VTool-R1 and OpenThinkIMG perform when given SciEviKit instead of their original tools?
4 comments
AuthorsOpenReview ↗

Response to Reviewer BMZ9(1/2)

Dear Reviewer BMZ9:

Thank you for your thoughtful comments and constructive suggestions! We address your concerns below.


W1/Q1: Chart Parser with Qwen3.5-4B

We agree that accurate chart reading is a core challenge in MSCV. Scientific figures vary widely in layout and visual encoding, making reliable numerical extraction difficult. We therefore adopt multimodal chart parsing as a simple and effective way to extract structured numerical evidence from diverse scientific figures.

To address this concern, we replaced the Astra-based chart parser with Qwen3.5-4B itself and repeated ToolSciVer training and evaluation using these self-generated parses. This variant uses no stronger external model for chart parsing. The results are in the following table.

SettingSciVerMuSciClaimsChart-type problem
ToolSciVer (Astra parser)84.07 ± 1.2780.79 ± 0.5283.31 ± 0.33
ToolSciVer (self-parsing)83.10 ± 0.6279.21 ± 0.9180.59 ± 0.31
VTool-R181.18 ± 0.0376.78 ± 0.7578.33 ± 0.45
OpenThinkIMG75.94 ± 0.2375.60 ± 0.9472.92 ± 0.65

Compared with Astra-parsing, self-parsing reduces overall accuracy by 0.97 and 1.58 percentage points on SciVer and MuSciClaims, with a larger decrease of 2.72 points on chart-type problems. Nevertheless, it still outperforms the strongest baseline(VTool-R1) with the same backbone by 1.92 and 2.43 points on the two benchmarks. These results show that a stronger parser contributes to performance, but ToolSciVer remains effective even when chart parsing is performed by the same small model. Its improvements on MSCV therefore do not require a stronger external chart parser.

For chart parse tool call ratio, Chart Parse accounts for 44.80% of all tool-call attempts in the original ToolSciVer with the Astra chart parser, compared with 21.69% in the variant trained with Qwen3.5-4B as the chart parser. One possible explanation is that the Qwen3.5-4B parser may provide less accurate chart evidence than Astra, reducing the reward benefit of invoking this tool during GRPO. This could encourage the learned policy to invoke Chart Parse less frequently.


W1/Q3: Baselines with SciEviKit

We conducted a toolset replacement experiment on Gemma4-E4B. We retained the training recipes of VTool-R1 and OpenThinkIMG, only replaced their original toolsets with SciEviKit, and updated the corresponding tool descriptions in the prompts. We then retrained both methods and evaluated them on SciVer and MuSciClaims. We then evaluated both methods on SciVer and MuSciClaims using the same SciEviKit backend and inference-time interaction budget as ToolSciVer. The results are in the following table.

MethodToolsetSciVer (%)MuSciClaims (%)
VTool-R1Original74.23 ± 1.1668.78 ± 0.80
VTool-R1SciEviKit77.20 ± 1.0071.16 ± 1.65
OpenThinkIMGOriginal72.40 ± 1.6171.16 ± 0.98
OpenThinkIMGSciEviKit77.63 ± 0.5870.76 ± 0.70
ToolSciVerSciEviKit81.20 ± 0.9574.65 ± 0.59

SciEviKit improves VTool-R1 by 2.97 and 2.38 percentage points on SciVer and MuSciClaims. OpenThinkIMG also improves substantially on SciVer, gaining 5.23 percentage points. These results show that SciEviKit benefits existing methods while retaining their training recipes, further validating the effectiveness of SciEviKit.

One point requiring clarification is OpenThinkIMG’s similar performance on MuSciClaims under the two toolsets. This is because MuSciClaims consists entirely of multi-panel scientific figures, for which zooming into relevant regions is the only tool functionality needed. OpenThinkIMG’s original toolset already includes a crop/zoom tool, providing this capability. Consequently, replacing its toolset with SciEviKit offers little additional functionality for these examples, resulting in comparable performance.

AuthorsOpenReview ↗

Response to Reviewer BMZ9(2/2)

W2/Q2: Checkpoint Selection & Multi-Seed Result

Yes, in the original experiments, checkpoints were selected based on test-set performance. To address your concern, we 1) clarify the fairness of the comparisons and 2) select the best checkpoint on a disjoint evaluation set and report its performance on the test set. We additionally conduct multi-seed training experiments with Qwen3.5-4B to assess the robustness of ToolSciVer’s performance across training seeds.

First, the same checkpoint-selection and reporting protocol was applied to ToolSciVer and all training-based baselines, including VTool-R1 and OpenThinkIMG. The reported comparisons therefore use a consistent selection criterion across these methods. Similar reporting practices also appear in prior work. For example, OPSD[1] reports the best score across periodically evaluated checkpoints and reports peak performance for its GRPO baseline. We nevertheless acknowledge that a consistent protocol and prior precedent alone do not address the concern about separating checkpoint selection from final reporting.

To further address your concern about this, we partitioned each original test set into disjoint evaluation and test subsets in a 1:2 ratio, selected the best checkpoints solely based on evaluation set performance, and reported their accuracy on the test set. Under this protocol, ToolSciVer achieves the highest mean overall accuracy in all six dataset–backbone combinations. On SciVer, it outperforms the strongest baseline by 4.35, 2.19, and 3.80 percentage points for Qwen3.5-4B, InternVL3.5-4B, and Gemma4-E4B, respectively. On MuSciClaims, the corresponding improvements are 0.20, 0.99, and 1.28 percentage points. Although the margins vary across settings, ToolSciVer consistently ranks first, supporting the robustness of our comparative conclusions under evaluation-based checkpoint selection.

BackboneMethodSciVer (%)MuSciClaims (%)
Qwen3.5-4BCoT w/o SciEviKit72.16 ± 7.1868.94 ± 3.12
Qwen3.5-4BChartAgent78.46 ± 0.8381.11 ± 0.86
Qwen3.5-4BDocLens61.56 ± 0.6274.98 ± 1.04
Qwen3.5-4BVTool-R180.41 ± 0.4876.85 ± 0.51
Qwen3.5-4BOpenThinkIMG76.11 ± 0.6275.17 ± 1.12
Qwen3.5-4BToolSciVer84.76 ± 1.5481.31 ± 1.07
InternVL3.5-4BCoT w/o SciEviKit65.74 ± 0.3270.62 ± 4.78
InternVL3.5-4BChartAgent54.92 ± 4.3565.68 ± 2.82
InternVL3.5-4BDocLens64.37 ± 0.3570.72 ± 2.11
InternVL3.5-4BVTool-R168.77 ± 1.1674.98 ± 0.86
InternVL3.5-4BOpenThinkIMG67.22 ± 0.6176.26 ± 0.89
InternVL3.5-4BToolSciVer70.96 ± 2.3977.25 ± 2.52
Gemma4-E4BCoT w/o SciEviKit77.11 ± 1.6673.59 ± 0.51
Gemma4-E4BChartAgent74.61 ± 1.6574.98 ± 1.49
Gemma4-E4BDocLens72.71 ± 1.6274.68 ± 2.15
Gemma4-E4BVTool-R175.31 ± 1.6270.72 ± 1.20
Gemma4-E4BOpenThinkIMG74.41 ± 1.4272.80 ± 2.08
Gemma4-E4BToolSciVer80.91 ± 1.1176.26 ± 1.57

We additionally conducted multi-seed training experiments with Qwen3.5-4B for ToolSciVer, VTool-R1, and OpenThinkIMG, reporting mean accuracy and standard deviation across training runs. ToolSciVer achieves the highest mean accuracy on both benchmarks, outperforming the strongest baseline, VTool-R1, by 3.80 and 2.77 percentage points on SciVer and MuSciClaims. These results support the robustness of ToolSciVer’s performance gains across training seeds.

MethodSciVer (%)MuSciClaims (%)
VTool-R180.97 ± 2.7477.03 ± 1.58
OpenThinkIMG75.43 ± 3.0276.50 ± 2.00
ToolSciVer84.77 ± 2.0679.80 ± 0.71

W3: Benefit of OPSD

We use OPSD as a warm-up stage to establish an initial tool-use policy before GRPO. It helps the model select relevant tools, generate executable calls with appropriate arguments, and use the returned observations for verification. Without this OPSD initialization, limited initial tool-use capability can lead GRPO to favor a direct-answering policy. In our experiments, GRPO without OPSD warm-up invokes tools on only 0.30% of SciVer examples and 0.59% of MuSciClaims examples. This policy restricts opportunities to acquire additional evidence through tools and limits the resulting performance gains. With a lower annotation burden than SFT warm-up, OPSD provides a starting policy that subsequent GRPO can refine for verification accuracy, tool-use validity, and efficiency.

Table 3 supports the benefit of this initialization. ToolSciVer outperforms tool-enabled GRPO without warm-up by 1.67 and 3.56 percentage points on SciVer and MuSciClaims, and exceeds no-tool GRPO by 3.50 and 2.90 points. These results support the value of establishing an initial tool-use policy before RL.


References:

[1] Zhao, S., et al. 2026. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. ICML 2026.

Thanks for the extensive rebuttal. Concerns addressed. Scores increased by 1.

Thank you for your feedback and positive assessment of our response! We are glad that our responses addressed your concerns, and we deeply appreciate your time and constructive suggestions.

Reviewer_xxAsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to improve multimodal claim verification for scientific tasks by teaching MLLMs to use tools to extract information from scientific figures. The paper makes two major contributions: (a) SciEviKit - a set of tools that the MLLM can use to extract relevant information, such as parsing tables and figures, and (b) a training recipe to teach the MLLM how to use these tools to verify claims. This recipe includes two phases: first, a warm start via on-policy self distillation (OPSD), followed by RL training via GRPO.

The authors evaluate their approach on two benchmarks, showing improvements over baselines such as GRPO without tools, prompting for tool-use without GRPO, etc; as well as propreitary models with tool-use. The ablations also demonstrate the utility of various design choices such as OPSD warm start, the various reward terms, etc.

Critical strengths
  • The paper tackles an important problem of scientific claim verification, which requires processing complex figures, tables, etc., along with text.
  • The proposed approach performs quite well compared to baselines.
  • The paper has several ablations and analyses to isolate the role of various design choices. In particular, they explain why it is important to have OPSD warm start and integrate tool use.
Critical weaknesses
  • The main concern I have is that this paper applies a fairly well-known recipe of teaching tool-use for multimodal tasks (like pixelreasoner, deepeyes) to the task of multimodal scientific claim verification. The approach does combine this recipe with some more components like OPSD, but those components are also based on existing approaches.
  • According to Section 4 and Appendix B.3, the paper selects the checkpoint that performs best on the benchmarks, which can introduce test-set selection bias. It might be better to select the checkpoint based on a separate validation set.
  • When you train the model on one dataset, how well does it generalize to other benchmarks (for the same task as well as other multimodal tasks)? Could you evaluate your trained models on some other benchmarks for scientific claim verification? Or train only on one of the benchmarks and evaluate on the other (and do this for both benchmarks)? It's not clear to what degree the gains depend on having "in distribution" training data, or if the learned capabilities are general.
Questions for authors
  • How well do the gains scale with model size, training data size, etc.? The experiments in the paper focus on 4B models, it would be useful to see other model scales and dataset sizes.
5 comments
AuthorsOpenReview ↗

Response to Reviewer xxAs(1/3)

Dear Reviewer xxAs,

Thank you for your thoughtful comments and constructive suggestions! We address your concerns below.


W1: Novelty Clarification

Thank you for raising this point. Our novelty includes both the structured visual evidence tools and the OPSD warm-up based training strategy.

First, compared with approaches centered on a limited set of visual operations (e.g., image cropping and video-frame selection in Pixel Reasoner, and crop-and-zoom in DeepEyes), SciEviKit additionally supports structured table and chart extraction, providing broader coverage of the heterogeneous visual evidence encountered in MSCV. To further validate the effectiveness of our SciEviKit, we conducted a toolset replacement experiment on Gemma4-E4B. We retained the training recipes of VTool-R1 and OpenThinkIMG, only replaced their original toolsets with SciEviKit, and updated the corresponding tool descriptions in the prompts. We then retrained both methods and evaluated them on SciVer and MuSciClaims. The results are here.

MethodToolsetSciVer (%)MuSciClaims (%)
VTool-R1Original74.23 ± 1.1668.78 ± 0.80
VTool-R1SciEviKit77.20 ± 1.0071.16 ± 1.65
OpenThinkIMGOriginal72.40 ± 1.6171.16 ± 0.98
OpenThinkIMGSciEviKit77.63 ± 0.5870.76 ± 0.70
ToolSciVerSciEviKit81.20 ± 0.9574.65 ± 0.59

SciEviKit improves VTool-R1 by 2.97 and 2.38 percentage points on SciVer and MuSciClaims. OpenThinkIMG also improves substantially on SciVer, gaining 5.23 percentage points. These results show that SciEviKit benefits existing methods while retaining their training recipes.

Second, our training recipe combines OPSD-based tool-call warm-up with verification- and efficiency-aware tool-use reinforcement learning. We introduce an OPSD-based warm-up stage for learning visual tool use in MSCV. Prior visual tool-learning frameworks, such as OpenThinkIMG and Pixel Reasoner, initialize tool-use policies through supervised fine-tuning on synthesized demonstration trajectories. Building on OPSD, our warm-up instead uses reference tool calls and their arguments as teacher-only privileged guidance. A frozen copy of the same base VLM provides token-level supervision along the student’s own tool-interaction trajectories. This establishes an initial tool-use policy without requiring complete annotated reasoning demonstrations. With the same toolset, adding this warm-up improves accuracy over GRPO without warm-up by 1.67/3.56 percentage points on SciVer/MuSciClaims (Table 3). Furthermore, we design an MSCV-oriented composite reward that jointly accounts for verification accuracy, tool-call validity, and efficiency. Beyond the terminal accuracy reward used in OpenThinkIMG, our objective explicitly rewards valid, successfully executed calls to tools compatible with the visual evidence and incorporates an efficiency factor adapted from OTC. This factor evaluates tool usage relative to the fewest calls observed among successful rollouts for the same example, encouraging economical evidence acquisition. Together, these signals guide the model to select appropriate tools and avoid redundant interactions while maintaining accurate verification. Table 4 supports this design: incorporating (seffs_{\mathrm{eff}}) improves accuracy while reducing both tool calls and generated tokens. We provide additional details on tool-call counts and verification-stage text-token accounting in our response to Reviewer eDv6 (W2/Q3).

AuthorsOpenReview ↗

Response to Reviewer xxAs(2/3)

W2: Checkpoint Selection / Multiseed training

To address your concern on checkpoint selection, we 1) clarify the fairness of the comparisons in our original experiments and 2) provide new results by selecting the best checkpoint on a disjoint evaluation set and reporting its performance on the test set.

First, the same checkpoint-selection and reporting protocol was applied to ToolSciVer and all training-based baselines, including VTool-R1 and OpenThinkIMG. The reported comparisons therefore use a consistent selection criterion across these methods. Similar reporting practices also appear in prior work. For example, OPSD[1] reports the best score across periodically evaluated checkpoints and reports peak performance for its GRPO baseline. We nevertheless acknowledge that a consistent protocol and prior precedent alone do not address the concern about separating checkpoint selection from final reporting.

To further address your concern about this, we partitioned each original test set into disjoint evaluation and test subsets in a 1:2 ratio, selected the best checkpoints solely based on evaluation set performance, and reported their accuracy on the test set. Under this protocol, ToolSciVer achieves the highest mean overall accuracy in all six dataset–backbone combinations. On SciVer, it outperforms the strongest baseline by 4.35, 2.19, and 3.80 percentage points for Qwen3.5-4B, InternVL3.5-4B, and Gemma4-E4B, respectively. On MuSciClaims, the corresponding improvements are 0.20, 0.99, and 1.28 percentage points. Although the margins vary across settings, ToolSciVer consistently ranks first, supporting the robustness of our comparative conclusions under evaluation-based checkpoint selection.

BackboneMethodSciVer (%)MuSciClaims (%)
Qwen3.5-4BCoT w/o SciEviKit72.16 ± 7.1868.94 ± 3.12
Qwen3.5-4BChartAgent78.46 ± 0.8381.11 ± 0.86
Qwen3.5-4BDocLens61.56 ± 0.6274.98 ± 1.04
Qwen3.5-4BVTool-R180.41 ± 0.4876.85 ± 0.51
Qwen3.5-4BOpenThinkIMG76.11 ± 0.6275.17 ± 1.12
Qwen3.5-4BToolSciVer84.76 ± 1.5481.31 ± 1.07
InternVL3.5-4BCoT w/o SciEviKit65.74 ± 0.3270.62 ± 4.78
InternVL3.5-4BChartAgent54.92 ± 4.3565.68 ± 2.82
InternVL3.5-4BDocLens64.37 ± 0.3570.72 ± 2.11
InternVL3.5-4BVTool-R168.77 ± 1.1674.98 ± 0.86
InternVL3.5-4BOpenThinkIMG67.22 ± 0.6176.26 ± 0.89
InternVL3.5-4BToolSciVer70.96 ± 2.3977.25 ± 2.52
Gemma4-E4BCoT w/o SciEviKit77.11 ± 1.6673.59 ± 0.51
Gemma4-E4BChartAgent74.61 ± 1.6574.98 ± 1.49
Gemma4-E4BDocLens72.71 ± 1.6274.68 ± 2.15
Gemma4-E4BVTool-R175.31 ± 1.6270.72 ± 1.20
Gemma4-E4BOpenThinkIMG74.41 ± 1.4272.80 ± 2.08
Gemma4-E4BToolSciVer80.91 ± 1.1176.26 ± 1.57

We additionally conducted multi-seed training experiments with Qwen3.5-4B for ToolSciVer, VTool-R1, and OpenThinkIMG, reporting mean accuracy and standard deviation across training runs. ToolSciVer achieves the highest mean accuracy on both benchmarks, outperforming the strongest baseline, VTool-R1, by 3.80 and 2.77 percentage points on SciVer and MuSciClaims. These results support the robustness of ToolSciVer’s performance gains across training seeds.

MethodSciVer (%)MuSciClaims (%)
VTool-R180.97 ± 2.7477.03 ± 1.58
OpenThinkIMG75.43 ± 3.0276.50 ± 2.00
ToolSciVer84.77 ± 2.0679.80 ± 0.71

References:

[1] Zhao, S., et al. 2026. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. ICML 2026.

AuthorsOpenReview ↗

Response to Reviewer xxAs(3/3)

W3: Generalization

Thank you for suggesting this additional evaluation. Section 5.3 (Table 7) of our manuscript reports ToolSciVer’s transfer performance on ChartQA, DVQA, and FigureQA. To further assess generalization within MSCV, we additionally evaluated the Qwen3.5-4B-based ToolSciVer and baselines on AIChartClaim[1], a new MSCV benchmark outside our training dataset distribution, without any additional training on this dataset.

As shown below, ToolSciVer achieves 91.53 ± 0.87% accuracy, the highest mean accuracy among all compared methods, outperforming the strongest baseline, VTool-R1, by 0.84 percentage points. These results complement our existing transfer evaluations by providing evidence of cross-dataset generalization within MSCV.

MethodAIChartClaim Accuracy (%)
Base (w/o tools)88.19 ± 1.97
ChartAgent85.83 ± 1.10
DocLens80.28 ± 2.29
VTool-R190.69 ± 1.05
OpenThinkIMG80.97 ± 0.48
ToolSciVer91.53 ± 0.87

Additionally, we compare ToolSciVer’s Macro-F1 with the AIChartClaim results reported in MEVER [1]. ToolSciVer achieves 91.65 ± 0.95 Macro-F1 without additional training on AIChartClaim, compared with 71.6 ± 0.7 reported for MEVER under the gold-evidence setting. We provide these published results for context.

MethodAIChartClaim Macro-F1 (%)
GPT-4o51.0 ± 1.7
UniChart69.3 ± 1.6
ChartGemma69.1 ± 0.4
ECENet70.0 ± 0.7
MEVER71.6 ± 0.7
ToolSciVer (Qwen3.5-4B)91.65 ± 0.95

Q1: Scaling with Model Size and Data Size

To examine whether ToolSciVer's benefits extend beyond the 4B models studied in our manuscript, we conducted additional comparisons using the larger Qwen3.5-9B backbone.

MethodSciVer (%)MuSciClaims (%)
CoT w/o SciEviKit74.80 ± 2.0377.23 ± 1.54
ChartAgent81.63 ± 1.0777.76 ± 1.49
DocLens70.20 ± 1.0872.81 ± 0.94
VTool-R179.50 ± 2.8773.20 ± 0.80
OpenThinkIMG81.60 ± 1.5580.46 ± 0.98
ToolSciVer85.77 ± 1.4582.18 ± 1.95

ToolSciVer achieves the highest mean accuracy on both benchmarks, outperforming the strongest baseline on each dataset by 4.14 and 1.72 percentage points on SciVer and MuSciClaims, respectively. Compared with the Qwen3.5-4B results in our manuscript, ToolSciVer’s accuracy also increases by 1.70 and 1.39 percentage points. These results show that ToolSciVer’s advantage persists with a larger backbone, extending our findings beyond the original 4B setting.

Regarding training-data scaling, further expansion is constrained by the limited labeled data available in the current benchmarks. Reallocating held-out examples to training would reduce test-set coverage and the statistical precision of evaluation. Alternatively, we trained ToolSciVer with Gemma4-E4B using 25%, 50%, 75%, and 100% of the available training data, while keeping the test sets fixed. The results are shown below.

Dataset25%50%75%100%
SciVer75.83 ± 0.9578.10 ± 1.3781.37 ± 2.0181.20 ± 0.95
MuSciClaim72.28 ± 1.4373.60 ± 0.9875.05 ± 1.7174.65 ± 0.59

ToolSciVer benefits from increasing the amount of training data. Accuracy improves consistently from the 25% to the 75% setting on both benchmarks. Compared with using 25% of the training data, using the full training set improves accuracy by 5.37 and 2.37 percentage points on SciVer and MuSciClaims. The 75% and 100% settings yield similar mean accuracy, suggesting that performance begins to plateau within the available data range. These results complement our model-size experiments by demonstrating the benefits of scaling training data. We believe that additional high-quality training data could further improve ToolSciVer’s ability to select appropriate visual tools and verify scientific claims.


References:

[1] Delvin Ce Zhang, Suhan Cui, Zhelin Chu, Xianren Zhang, and Dongwon Lee. 2026. MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrieval. EACL 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal and Revision Summary

Dear AC and Reviewers,

Thank you for your thoughtful reviews and constructive suggestions. Your feedback has helped us improve the manuscript through additional experiments, more systematic analyses, and clearer explanations. This has helped us improve our manuscript summarized as below.

  • SciEviKit’s advantage across different training recipes. We added comparisons with VTool-R1 and OpenThinkIMG retrained using SciEviKit, showing stronger performance on both benchmarks (Section 5.2, Table 4).
  • More detailed analysis of OPSD benefits. Without OPSD, tool-enabled GRPO invokes tools on only 0.30% of SciVer and 0.59% of MuSciClaims examples (Section 5.2, “Warm-up strategy”).
  • ToolSciver’s efficiency at the system level. Our expanded analysis shows lower total inference-time token usage than VTool-R1 and OpenThinkIMG (Section 5.2, Table 5; Appendices B.7 and C.4).
  • ToolSciVer generalizes across datasets and tasks. Our expanded evaluation demonstrates transfer to out-of-domain scientific claim verification and broader chart and figure reasoning tasks (Section 5.3; Appendix C.8).

We have provided the revised manuscript, with revisions highlighted in blue. We welcome any further feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.