acceptodds
Under review as a conference paper at ICLR 2027

VisProve: Learning Typed Visual Provenance for Scientific Document Question Answering

Abstract

Visual citations show where an answer is supported, but not how the cited content contributes to it. In scientific document question answering, a model may read a value directly, compress several observations, or combine evidence to infer a conclusion. We study typed visual provenance, where a system generates an answer together with source regions and explicit support relations. We introduce **ReViE**, a dataset of 3,941 questions over complete scientific documents, with page, section, and region annotations labeled as *Quotation*, *Compression*, or *Inference*. We further propose **VisProve**, which augments joint answer and provenance generation with differentiable Spatial Evidence Reading and Evidence–Answer Relation Feedback. VisProve conditions next-token prediction on localized visual evidence and its estimated support relation through a gated norm-bounded residual. On the document-disjoint ReViE test set, VisProve improves Typed-F1 from 3.52 to 5.48 and Section-F1 from 45.99 to 53.24 over joint supervised fine-tuning with the same backbone, while Page-F1 changes only marginally. These results suggest that coupling visual grounding with support relation modeling mainly improves fine-grained provenance, while the low absolute Typed-F1 reveals a substantial gap between answer generation and precise visual evidence attribution.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
scientific document question answering, fine-grained visual provenance, visual evidence grounding, evidence-answer relations, vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_shGmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper tackles the task of fine-grained visual provenance. Given a question and a multi-page document, the task requires generating an answer along with supporting evidence regions at different spatial scopes (page, section, block) and relation types (Quotation, Compression, Inference). A model called VisProve is introduced for the task. This model adds Spatial Evidence Reading (SER) and Evidence-Answer Relation Feedback (ERF) modules to a pretrained VLM. The paper also introduces ReViE, a benchmark for scientific document QA created by augmenting an existing benchmark. Experiments show the relatively superior performance of VisProve over other baselines and ablative variants, especially for section localization and relational types.

Critical strengths

The paper identifies a nice complementary gap - a set of existing works address provenance types but do not provide grounding, while other set of existing works have the opposite shortcoming. Providing visual evidence for generated answers is useful, and explicitly characterizing how the evidence supports an answer is an interesting direction.

The proposed SER and ERF modules are meaningful architectural additions. The ablations suggest that these components improve fine-grained provenance prediction over the SFT baseline. In particular, the improvements in Section-F1 and Typed-F1 are encouraging, even though the absolute performance remains low.

The augmentation of existing dataset can also be counted as a good effort aligned with the intended task.

Critical weaknesses

My main concern is with the task formulation itself.

The approach generates an answer and a set of evidence regions, with each region assigned a relation type. However, the evidence regions are associated with the complete answer rather than the specific parts of the answer they support (lines 189-194).

This formulation seems inadequate for the stated objective of fine-grained provenance. An answer can contain multiple statements, each supported by different evidence regions and potentially involving different relation types. For instance, the answer shown in Fig. 4 contains multiple statements. The current representation does not explicitly indicate which evidence element supports which sentence within the answer.

If this system were to be deployed, a user would still need to inspect the evidence and determine how different parts of the answer are supported. This defeats an important part of the motivation for the task. The provenance should be directly associated with individual answer spans or statements, possibly through interleaved answer and evidence generation, as explored in related grounding approaches (e.g. Venna2026).

The relation taxonomy also needs better motivation. Quotation, Compression, and Inference are adopted from prior work (Zhu et al., 2025; Wei et al., 2026b), but it is not clear what is different in their interpretation when applied to visual evidence. Apart from the limited examples provided, I found it difficult to understand the diversity of outputs expected under this formulation.

  1. Experimental validation and practical utility

The experimental results show improvements over the SFT baseline, but I am not convinced that they establish the practical effectiveness of the proposed approach.

The absolute Typed-F1 is only 5.48, despite improving over the SFT baseline of 3.52. Region-F1 is also low at 9.68. These results indicate that accurate fine-grained provenance remains largely unsolved by the proposed approach. It is difficult to see how useful the generated provenance would be in practice at this level of performance.

The authors compare against direct SFT and existing document grounding methods. However, I would also have liked to see comparisons with simpler modular approaches, such as:

Multi-page QA with visual grounding, followed by relation-type classification for each grounded region.

Existing relation-aware provenance methods adapted to produce spatial grounding.

These would help establish whether joint training and the proposed modules are necessary for the task.

The construction of ReViE also relies on GPT-5.6 Sol for evidence refinement and relation annotation, followed by human verification. It would have been useful to quantify the contribution of human verification and, if feasible, evaluate the model used during dataset construction as a reference point.

Some minor concerns:

The term "memory" used for the spatial visual representations in SER is confusing. These appear to be intermediate visual features extracted from the document rather than a memory mechanism in the usual sense. While the terminology is sometimes used for stored visual features, I do not see what is gained by calling them memory here.

Human evaluation (Fig. 5): The reported Pearson correlation of 0.9898 is computed over 11 model-level averages, although 220 individual answers were rated. This shows that the mean automated scores correlate strongly with mean human scores across models. It does not establish agreement at the individual-answer level. Since the automated judge evaluates individual answers, reporting answer-level correlation and agreement would be more informative. The current result cannot be definitively interpreted as validation of individual answer scoring.

Dataset size: ReViE contains 3,941 QA instances, with 3,115 used for training. Given the complexity of the task and the diversity of evidence types, I am not convinced this is sufficient supervision. The paper could discuss this limitation more explicitly.

Questions for authors

My principal concern is with the task formulation, particularly the lack of explicit correspondence between answer spans and supporting evidence. I do not think additional experiments on the current formulation would materially change my assessment.

I would therefore be fine if the authors prefer to prioritize addressing other reviewers' comments rather than spending substantial effort responding to mine.

2 comments
AuthorsOpenReview ↗

Clarifying Visual Provenance and Strengthening Empirical Validation

Thank you for your constructive review, which helped clarify our task and strengthen validation.

1. Region-level and span-level attribution

ReViE links evidence to the full answer. Each region has page, section, box, and relation annotations. Fine granularity concerns visual localization and support type. Answer-span alignment is complementary. In Fig. 9, the correct answer (Natural Padding) is linked to an incorrect region near a chart. Exact span alignment alone cannot resolve this grounding error.

Answer formReViETest409
Short/single sentence3,513/3,941 (89.14%)374/409 (91.44%)

Of these concise answers, 2,106 require multiple regions. We agree that span-level attribution is a valuable extension, especially for multi-statement answers whose spans rely on distinct regions (Fig. 4). It would precisely align answer components with visual evidence and will be a key focus of future work. We will clarify this complementary setting and our current scope in the revision.

2. Visual meaning of relations

Following TROVE and GenProve, labels describe how a visual region supports the answer.

RelationMeaningExample
QuotationReading a visible fact92% in a table
CompressionSummarizing explicit visual informationWorkflow steps
InferenceDeriving a conclusion through reasoningComparing two bars

One chart can support Quotation (reading a value) or Inference (comparing values). Multiple regions alone do not imply Inference. We will add examples.

3. Fine-grained evaluation

Typed-F1 = 5.48 jointly evaluates page, section, box, and relation matching. Usable-citation percentage is a separate quantity. Inputs average 18.28 pages and 2.16 evidence regions per QA. Results use the same backbone and training data.

MetricDefault SFTVisProve
Page-F175.0075.73
Section-F145.9953.24
Region-F15.919.68
Typed-F13.525.48

Region-F1 is conditioned on exact page-set matching. Table 1 general-purpose VLMs reach at most 7.08 Region-F1 and 3.21 Typed-F1. Appendix D illustrates evidence errors missed by answer-only scores.

4. Ground-then-classify baseline

Our modular baseline predicts answers/regions using Default SFT, then classifies relations with a ReViE-trained two-layer MLP on frozen Qwen3-VL-8B features. Inference uses predicted boxes and cannot revise them.

MetricSFTTwo-stageVisProve
Region-F15.915.919.68
Relation-F15.225.559.22
Typed-F13.523.765.48
Full-test Relation-F13.733.966.81

VisProve exceeds Two-stage by 3.77 Region-F1, 3.67 Relation-F1, and 1.72 Typed-F1 points. Table 3 compares existing methods under a shared output format.

5. Human verification

Two external annotators independently relabeled 180 balanced regions, blinded to original labels. Agreement was 150/180 (83.33%), Cohen's κ = 0.750, with positive agreement of 83.48% / 78.26% / 87.69% for Quotation / Compression / Inference. After third-annotator review of 42 disputes, 163/180 (90.56%) original labels had two independent supporting votes in this sample.

GPT-5.6 Sol assisted evidence refinement and relation labeling during construction. End-to-end QA requires predicting answers and evidence from full documents. Our comparison uses the same Qwen3-VL-8B backbone and SFT training data.

6. SER terminology

SER organizes each page's visual features into spatially encoded 16 × 16 grids. We will replace “memory” with “Visual Feature Bank” in Section 4.2 and Appendix B.2 to avoid ambiguity.

7. LLM-as-Judge agreement

Fig. 5's r = 0.9898 uses 11 model-level means. For item-level agreement, we compared 220 original model–question ratings (11 models × 20 questions, 197 unique answers) with median human scores on a 0–5 scale, without rescoring.

MeasureValue95% CI
Pearson r0.9323[0.8746, 0.9699]
Spearman ρ0.9174[0.8633, 0.9519]
MAE0.3682[0.2295, 0.5319]
Exact agreement71.36%[61.36%, 80.91%]
Error ≤1 point94.09%[89.55%, 97.73%]
Mean Judge − Human+0.0045[−0.1705, 0.1591]

CIs use 3,000 question-clustered bootstrap resamples. The Judge scores answer text only and is excluded from Overall.

8. Dataset scale and supervision

ReViE spans eight domains with document-disjoint splits, 3,941 QAs (3,115 training), and 8,523 typed evidence regions (6,738 training).

MetricZero-shot Qwen3-VL-8BDefault SFTVisProve
ROUGE-L48.2764.2467.08
Section-F131.9045.9953.24
Typed-F10.183.525.48

Held-out gains demonstrate the value of structured supervision. We will clarify supervision density in the revision.

We hope the modular baseline, human verification, and item-level agreement analysis strengthen our findings and address your concerns. We respectfully ask you to reconsider your assessment.

Reviewer_shGmOpenReview ↗

Retain my assessment

I read the response. It clarifies a few minor points. However, as mentioned in my original review, my disagreement is fundamentally with the way the task is posed and how it does not serve practical needs of grounded verification systems at the required granularity. I choose to retain my score.

Reviewer_xSiVOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies typed visual provenance for scientific document question answering. Given a question and the complete, ordered sequence of document page images, the model generates an answer and evidence items specifying a page, section, bounding box, and support relation: Quotation, Compression, or Inference. The authors construct ReViE from SciEGQA, comprising 3,941 QA instances and 8,523 evidence regions from 1,612 documents, with document-disjoint splits. They further propose VisProve, which freezes a jointly fine-tuned Qwen3-VL-8B backbone and conditions generation on spatial evidence reading and evidence–answer relation feedback through a gated, norm-bounded residual. On the 409-instance test set, VisProve improves Typed-F1 from 3.52 to 5.48 and Section-F1 from 45.99 to 53.24 over Default SFT, with a small change in Page-F1. The work introduces a useful task and data resource and reports initial improvements in fine-grained provenance, but the reliability of the relation annotations and the attribution of the methodological gains require stronger validation.

Critical strengths

S1. The task meaningfully combines visual source localization with evidence-use relations. The paper asks models not only to locate supporting regions, but also to distinguish direct reading, compression, and inference. This makes a dimension of evidence use that page references or bounding boxes alone cannot express explicitly evaluable, which is relevant to the verifiability of scientific document QA. Although the relation categories build on textual provenance research, combining them with region-level visual evidence in scientific documents is a useful extension.

S2. Full-document inputs and document-disjoint splits make the dataset a useful research resource. ReViE contains 1,612 documents and 8,523 evidence regions and includes multi-region, multi-page, and multi-hop instances. Models must locate sparse evidence within complete documents rather than operate only on preselected gold evidence pages. The document-disjoint split also prevents instances from the same source document from being directly distributed across training and testing. These choices provide a useful basis for studying evidence localization under challenging document inputs.

S3. The shared-backbone comparison provides initial evidence of improved fine-grained provenance, with appropriately qualified conclusions.Relative to Default SFT, the full model improves Typed-F1 from 3.52 to 5.48 and Section-F1 from 45.99 to 53.24, providing initial evidence for the effectiveness of the complete training configuration. The authors also acknowledge the low absolute provenance scores and the nonuniform improvements in answer quality, and explicitly distinguish conditional region metrics from full-test-set Typed-F1. These qualifications help readers interpret the scope of the results.

Critical weaknesses

W1. The variants in Table 8 change not only the feedback components, but also the trainable parameters, auxiliary supervision positions, field-level cross-entropy coverage, and associated regularization. For example, “w/o SER” retains a frozen spatial reader, while the full model and “w/o ERF” use different localization supervision positions and field losses. Table 2 therefore primarily compares training configurations, rather than cleanly separating the contributions of spatial reading, relation feedback, and additional supervision. The full model also receives additional component training relative to Default SFT, without a simple control matched for training budget and field losses. This is my main concern about the methodological contribution.

W2. The paper describes human review and second-round expert validation, but does not report the review coverage, the extent of independent duplicate annotation, agreement by relation category, or the rate at which humans corrected model-generated labels. This is particularly important because Compression accounts for only 2.84% of evidence regions, and mixed support relations are reduced to a single label using a priority rule. The human evaluation in Section 5.5 assesses answer text only and cannot validate visual evidence or relation annotations. Since these labels are a central addition in ReViE, their reliability needs more direct evidence.

W3. Evidence-IoU, Region-F1, and Relation-F1 are computed on each model’s own subset of instances with exactly correct evidence-page sets, but the subset sizes are not reported. Cross-model comparisons may therefore involve different evaluation populations. Relation-F1 also depends on spatial overlap, and Typed-F1 additionally requires section agreement, so improvements in these metrics do not independently establish improved relation identification. Full-test-set Typed-F1 is a useful end-to-end result, but a decomposition is needed to determine whether the gains primarily arise from localization, section prediction, or relation modeling.

Questions for authors

1.Could you provide a comparison that controls auxiliary supervision positions, field losses, training epochs, and input budgets while varying only spatial and relation feedback? Please also clarify the initialization or training source of the frozen spatial reader, gate, and up-projection in “w/o SER.” Does the full configuration retain its Typed-F1 advantage after controlling these factors?

2.How many instances or regions received human review, and how many were independently annotated more than once? What are the agreement and human correction rates for each relation category? In particular, what independent annotation evidence supports a stable distinction between Compression and Inference?

3.How many test instances satisfy the exact-page-set condition for each model? Could you additionally report region and relation matching over the full test set, together with relation classification given gold evidence regions or an equivalent analysis, to distinguish the contributions of localization, section prediction, and relation identification to the final gains?

4.Could you report paired confidence intervals for VisProve versus Default SFT, accounting for dependence among instances from the same document? Please also provide test counts and results for each relation category. Do the improvements extend to Inference, Compression, and instances involving figure or table evidence?

2 comments
AuthorsOpenReview ↗

Controlled Ablations, Label Reliability, and Statistical Validation

Thank you for your careful and constructive review. Your comments helped us strengthen experimental controls, clarify evaluation protocols, and provide further evidence for our claims.

W1/Q1. Isolating SER and ERF Contributions

The original Table 8 varied training settings alongside feedback paths, limiting attribution. We conducted a matched ablation with identical SFT backbone, module initialization, 3,115 training QAs, two-epoch schedule, trainable components, auxiliary positions, losses, regularization, and visual budget. Only decoder feedback differed.

Feedback (Test409)Section-F1Region-F1Relation-F1Typed-F1
Spatial + relation (st+ets_t + e_t)53.409.769.355.52
Relation only (ete_t)49.188.027.624.18
Spatial only (sts_t)48.907.887.423.86

Joint feedback improves Typed-F1 by 1.34 and 1.66 points over relation-only and spatial-only feedback, respectively. These runs are distinct from Table 2. In the original w/o SER configuration, the frozen reader, gate, and up-projection inherit parameters from a trained provenance-module checkpoint. The reader supplies features to ERF, with direct spatial feedback to the decoder disabled. We will clarify this distinction and initialization in Appendix B.2.

W2/Q2. ReViE Relation-Label Reliability

Two external annotators independently relabeled 180 regions from 180 distinct QAs and documents, blinded to original labels and each other's annotations. The sample contained 60 regions each for Quotation, Compression, and Inference.

Independent auditResult
Two-annotator agreement150/180 (83.33%)
Cohen's κ0.750
Positive agreement, Q / C / I83.48% / 78.26% / 87.69%
Agreement with originals, annotators 1 / 286.11% / 83.89%

A third blinded annotator reviewed 42 disputes (30 between the first two annotators and 12 where both disagreed with the stored label). Overall, 163/180 (90.56%) original labels received support from at least two independent annotators. The third annotator supported the original label in 10/11 direct Compression–Inference disputes.

Compression summarizes explicit information, whereas Inference derives an additional conclusion (e.g., by comparing values). Multiple evidence regions alone do not imply Inference. The 90.56% rate applies to this class-balanced sample. We will report the annotation protocol and adjudication details.

W3/Q3. Localization and Relation Identification

We rescored all 409 test QAs, retaining page consistency, IoU-based matching, and one-to-one assignment. This extends evaluation beyond each model's exact-page-set-correct subset. We also tested relation classification on identical gold regions, withholding gold relation labels.

EvaluationMetricDefault SFTVisProve
Full Test409Exact-page-set correct282/409293/409
Full Test409Region-F14.227.15
Full Test409Relation-F13.736.81
Gold evidenceAccuracy (%)72.676.2
Gold evidenceRelation Macro-F1 (%)43.147.8

The 2.93-point Region-F1 gain is independent of relation-label agreement. With localization fixed by gold evidence, Relation Macro-F1 improves by 4.7 points. Original Section-F1 (45.99 → 53.24) and end-to-end Typed-F1 (3.52 → 5.48) further support improvements in both localization and relation identification.

Q4. Statistical Stability and Evidence Categories

We performed 3,000 document-level paired bootstrap resamples over 155 test documents, keeping each document's QAs together.

MetricSFT → VisProveGain (95% CI)
Typed-F13.52 → 5.48+1.96 [1.71, 2.21]
Section-F145.99 → 53.24+7.25 [6.00, 8.39]
Region-F15.91 → 9.68+3.77 [1.94, 5.64]
Relation-F15.22 → 9.22+4.00 [2.16, 6.08]

All confidence intervals exclude zero. Region-F1 and Relation-F1 here use the paper's exact-page-set-conditioned protocol. Full-test metrics are reported above.

Gold-evidence relationQAs / RegionsF1, SFT → VisProve
Quotation338 / 63383.5 → 86.0
Compression17 / 296.5 → 10.5
Inference112 / 20239.4 → 47.0

QAs may involve multiple relation types. Inference improves by 7.6 points. The Compression estimate uses fewer instances.

Visual evidenceQAsTyped-F1, SFT → VisProveRegion-F1, SFT → VisProve
Figure893.80 → 5.906.40 → 9.80
Table203.10 → 3.505.20 → 5.60
Mixed353.30 → 4.505.60 → 7.40

All three groups improve, with relatively modest table gains. Grouped Region-F1 uses the exact-page-set-conditioned protocol. We will include these analyses in the revised manuscript.

We hope these controlled experiments, independent annotation checks, and statistical analyses address your concerns and strengthen our main claims. We appreciate your consideration and respectfully ask you to reconsider your assessment in light of these findings.

We hope the above clarifications and theadditional experiments sufficiently addressed your concerns. If you aresatisfied, we kindly request you to considerupdating the score to reflect the newly addedresults and discussion. We remain committed toaddressing any remaining points you may haveduring the discussion phase.

Reviewer_GkrjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes two interrelated research components. 1) ReViE: A dataset of 'typed' visual provenance for QA on scientific documents. This dataset requires models to specify not only 'where' the evidence for an answer is located but also 'how' that evidence supports the answer. 2) VisProve: A methodology that extracts visual features from evidence regions and utilizes feedback to predict the relationship between the evidence and the answer. In ReViE, given a question and all document page images, the model generates an answer along with a list of evidence items. Each evidence item is annotated with information regarding its page, section, bounding box, and relation type. To construct ReViE, the authors extended the existing SciEGQA dataset. The relationship labels were initially assigned using GPT-5.6 and subsequently reviewed through human review. VisProve employs SFT on the Qwen3-VL-8B model, incorporating two additional modules: 1) SER extracts visual features from evidence regions; 2) ERF predicts the relationship between the evidence and the answer. For evaluation, the authors introduced new F1-score family metrics.

Critical strengths
  1. The motivation for distinguishing between the location of evidence and how that evidence supports an answer is clear. This task integrates two research areas of visual grounding and textual provenance.
  2. In this setup, the entire document images are used as input, which more faithfully reflects real-world usage scenarios.
  3. This paper does not overstate its findings. The authors acknowledge that the degree of improvement in answer quality varies depending on the evaluation metric used.
Critical weaknesses
  1. Scientific rigor is questionable. The task (typed visual provenance), relation labels (ReViE), evaluation metrics (Relation-F1, Typed-F1, Overall, the Judge prompt), and method are all defined by the authors and evaluated only within this closed framework, and the reviewer cannot access the data as anonimized link is not provided. Apart from ROUGE-L and BERTScore, none of the metrics cite prior work.
  2. ERF in VisProve is trained to predict the relation labels, and its performance is measured with metrics based on the same labels (Relation-F1, Typed-F1). It is therefore impossible to tell whether the improvement reflects a genuine gain in provenance ability or fitting to the label distribution.
  3. For ReViE, the reliability of these labels is not convincing to me. The paper only states that they went through human annotator review and second-round expert validation ,which looks insufficient. Authors should follow the conventional data publication scheme [1] when having it as a contribution of this work. Such evidence should be part of the submission. If it is provided only during the rebuttal phase, it cannot be properly reviewed given the limited time. I recommend to publish their dataset in parallel with their method in a separate work.
  4. Comparisons outside this framework are also weak. Table 1 compares VisProve, trained on ReViE, against zero-shot VLMs, and the prior methods in Table 3 (VISA, CoE, UniDoc-RL) are evaluated without training on ReViE. The only comparison under the same conditions is Default SFT with the same backbone, and VisProve is not evaluated on any benchmark other than ReViE. Since both ReViE and VisProve depend on labels that cannot be verified, neither contribution is sufficiently supported.

[1] NeurIPS 2026 Dataset and Code Submission https://neurips.cc/Conferences/2026/CallForEvaluationsDatasets

Questions for authors
  1. What evidence supports the validity of this work outside the authors' own task, data, and metrics?
  2. How can the authors show that the improvement of VisProve reflects a genuine gain in provenance ability rather than fitting to the relation label distribution?
  3. Why was no evidence of the relation label quality included in the submission?
2 comments
AuthorsOpenReview ↗

Clarifying Task Foundations, Independent Validation, and Fair Evaluation

We thank the reviewer for the thoughtful assessment and for recognizing our motivation. Your comments have helped us clarify the evidence for ReViE and VisProve.

W1/Q1. Task foundations and metrics

ReViE combines visual grounding (VISA, SciEGQA) with evidence-use provenance (TROVE, GenProve). Each region has page, section, box, and support-relation labels.

DimensionMetricsPrinciple
AnswerROUGE-L, BERTScore-F1Text similarity
LocalizationPage-F1, Section-F1Set matching
LocalizationEvidence-IoU, Region-F1IoU, one-to-one matching
Typed provenanceRelation-F1, Typed-F1Relation-aware matching

We use IoU (Everingham et al., 2010) and Hungarian matching (Kuhn, 1955). Appendices C.1–C.2 define matching and validity rules. Overall equally weights answer, localization, and typed provenance (C.3). The 0–5 Judge scores answers only (C.4, Fig. 8), is excluded from Overall, and correlates with human ratings (r=0.9323, 220 observations). Gains also occur without relation matching. Section-F1 rises from 45.99 to 53.24, Evidence-IoU from 5.07 to 8.58, and Region-F1 from 5.91 to 9.68 (last two conditioned on exact page sets). We will add these citations.

W2/Q2. Evidence beyond relation-label frequencies

We evaluated localization on all 409 test QAs and relation classification on identical gold regions, withholding gold relation labels.

Metric / settingDefault SFTVisProve
Exact-page-set correct282/409293/409
Full-test Region-F14.227.15
Full-test Relation-F13.736.81
Gold-evidence accuracy (%)72.676.2
Gold-evidence Macro-F1 (%)43.147.8
Gold-evidence Inference-F1 (%)39.447.0

Full-test Region-F1 gains 2.93 points without relation matching. Always predicting Quotation on the 864 gold regions would yield 28.19 Macro-F1 / 0 Inference-F1, versus VisProve's 47.8 / 47.0. A matched ablation yields 5.52 Typed-F1 with both feedback paths, versus 4.18 (relation only) and 3.86 (spatial only), controlling initialization, supervision, losses, trainable parameters, and budget. These results support gains beyond majority-class prediction.

W3/Q3. Independent annotation, access, and dataset contribution

The paper describes human review (Section 3.2, Appendices A.2–A.4). Two external annotators independently relabeled 180 regions (60 per class) from distinct QAs/documents, blinded to original labels.

Independent auditResult
Two-annotator agreement150/180 (83.33%)
Cohen's κ0.750
Positive agreement, Q / C / I83.48% / 78.26% / 87.69%
Agreement with originals, annotators 1 / 286.11% / 83.89%
Original labels with ≥2 independent votes163/180 (90.56%)

A third blinded annotator reviewed 42 disputes and supported the original label in 10/11 direct Compression–Inference disagreements. The 90.56% rate refers to this class-balanced sample. We will report the audit protocol.

We deferred a data/code link pending double-blind identity checks. Upon acceptance, we will release ReViE, VisProve code, and the evaluator. ReViE defines the benchmark, and VisProve studies how to learn it.

W4/Q1. Fair baselines and broader validity

ExperimentPurpose
Table 1, general VLMsCharacterize the task
Table 2, Default SFT vs. VisProveSame backbone/data, main method comparison
Table 3, VISA, CoE, UniDoc-RLCompare shared output metrics

Table 3 reports Overall_shared = 55.98 for VisProve versus 39.98 for UniDoc-RL. Table 2 and controlled tests support method attribution.

Our two-stage modular baseline uses Default SFT for answers/regions and a trained two-layer MLP on frozen Qwen3-VL-8B features for relation classification. Inference uses predicted boxes and cannot revise their locations.

MetricDefault SFTTwo-stageVisProve
Region-F15.915.919.68
Relation-F15.225.559.22
Typed-F13.523.765.48
Full-test Relation-F13.733.966.81

Region-F1 and Relation-F1 use the paper's exact-page-set condition. Full-test Relation-F1 covers all 409 QAs. Post-hoc classification cannot revise grounding. VisProve improves both grounding and classification. SciEGQA lacks support-relation labels and ReFInE lacks visual boxes, preventing direct full Typed-F1 comparison. We will clarify the comparison scope.

We hope these results address your concerns about scientific rigor, label reliability, and fair evaluation. We would be grateful if you would reconsider your assessment and rating.

My concerns remain after the rebuttal so that I will maintain my score. Thank you authors for your effort, and I recommend to resubmit your work into two different tracks.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.