- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper tackles the task of fine-grained visual provenance. Given a question and a multi-page document, the task requires generating an answer along with supporting evidence regions at different spatial scopes (page, section, block) and relation types (Quotation, Compression, Inference). A model called VisProve is introduced for the task. This model adds Spatial Evidence Reading (SER) and Evidence-Answer Relation Feedback (ERF) modules to a pretrained VLM. The paper also introduces ReViE, a benchmark for scientific document QA created by augmenting an existing benchmark. Experiments show the relatively superior performance of VisProve over other baselines and ablative variants, especially for section localization and relational types.
Critical strengths
The paper identifies a nice complementary gap - a set of existing works address provenance types but do not provide grounding, while other set of existing works have the opposite shortcoming. Providing visual evidence for generated answers is useful, and explicitly characterizing how the evidence supports an answer is an interesting direction.
The proposed SER and ERF modules are meaningful architectural additions. The ablations suggest that these components improve fine-grained provenance prediction over the SFT baseline. In particular, the improvements in Section-F1 and Typed-F1 are encouraging, even though the absolute performance remains low.
The augmentation of existing dataset can also be counted as a good effort aligned with the intended task.
Critical weaknesses
My main concern is with the task formulation itself.
The approach generates an answer and a set of evidence regions, with each region assigned a relation type. However, the evidence regions are associated with the complete answer rather than the specific parts of the answer they support (lines 189-194).
This formulation seems inadequate for the stated objective of fine-grained provenance. An answer can contain multiple statements, each supported by different evidence regions and potentially involving different relation types. For instance, the answer shown in Fig. 4 contains multiple statements. The current representation does not explicitly indicate which evidence element supports which sentence within the answer.
If this system were to be deployed, a user would still need to inspect the evidence and determine how different parts of the answer are supported. This defeats an important part of the motivation for the task. The provenance should be directly associated with individual answer spans or statements, possibly through interleaved answer and evidence generation, as explored in related grounding approaches (e.g. Venna2026).
The relation taxonomy also needs better motivation. Quotation, Compression, and Inference are adopted from prior work (Zhu et al., 2025; Wei et al., 2026b), but it is not clear what is different in their interpretation when applied to visual evidence. Apart from the limited examples provided, I found it difficult to understand the diversity of outputs expected under this formulation.
- Experimental validation and practical utility
The experimental results show improvements over the SFT baseline, but I am not convinced that they establish the practical effectiveness of the proposed approach.
The absolute Typed-F1 is only 5.48, despite improving over the SFT baseline of 3.52. Region-F1 is also low at 9.68. These results indicate that accurate fine-grained provenance remains largely unsolved by the proposed approach. It is difficult to see how useful the generated provenance would be in practice at this level of performance.
The authors compare against direct SFT and existing document grounding methods. However, I would also have liked to see comparisons with simpler modular approaches, such as:
Multi-page QA with visual grounding, followed by relation-type classification for each grounded region.
Existing relation-aware provenance methods adapted to produce spatial grounding.
These would help establish whether joint training and the proposed modules are necessary for the task.
The construction of ReViE also relies on GPT-5.6 Sol for evidence refinement and relation annotation, followed by human verification. It would have been useful to quantify the contribution of human verification and, if feasible, evaluate the model used during dataset construction as a reference point.
Some minor concerns:
The term "memory" used for the spatial visual representations in SER is confusing. These appear to be intermediate visual features extracted from the document rather than a memory mechanism in the usual sense. While the terminology is sometimes used for stored visual features, I do not see what is gained by calling them memory here.
Human evaluation (Fig. 5): The reported Pearson correlation of 0.9898 is computed over 11 model-level averages, although 220 individual answers were rated. This shows that the mean automated scores correlate strongly with mean human scores across models. It does not establish agreement at the individual-answer level. Since the automated judge evaluates individual answers, reporting answer-level correlation and agreement would be more informative. The current result cannot be definitively interpreted as validation of individual answer scoring.
Dataset size: ReViE contains 3,941 QA instances, with 3,115 used for training. Given the complexity of the task and the diversity of evidence types, I am not convinced this is sufficient supervision. The paper could discuss this limitation more explicitly.
Questions for authors
My principal concern is with the task formulation, particularly the lack of explicit correspondence between answer spans and supporting evidence. I do not think additional experiments on the current formulation would materially change my assessment.
I would therefore be fine if the authors prefer to prioritize addressing other reviewers' comments rather than spending substantial effort responding to mine.
2 comments
Retain my assessment
I read the response. It clarifies a few minor points. However, as mentioned in my original review, my disagreement is fundamentally with the way the task is posed and how it does not serve practical needs of grounded verification systems at the required granularity. I choose to retain my score.
Clarifying Visual Provenance and Strengthening Empirical Validation
Thank you for your constructive review, which helped clarify our task and strengthen validation.
1. Region-level and span-level attribution
ReViE links evidence to the full answer. Each region has page, section, box, and relation annotations. Fine granularity concerns visual localization and support type. Answer-span alignment is complementary. In Fig. 9, the correct answer (Natural Padding) is linked to an incorrect region near a chart. Exact span alignment alone cannot resolve this grounding error.
Of these concise answers, 2,106 require multiple regions. We agree that span-level attribution is a valuable extension, especially for multi-statement answers whose spans rely on distinct regions (Fig. 4). It would precisely align answer components with visual evidence and will be a key focus of future work. We will clarify this complementary setting and our current scope in the revision.
2. Visual meaning of relations
Following TROVE and GenProve, labels describe how a visual region supports the answer.
One chart can support Quotation (reading a value) or Inference (comparing values). Multiple regions alone do not imply Inference. We will add examples.
3. Fine-grained evaluation
Typed-F1 = 5.48 jointly evaluates page, section, box, and relation matching. Usable-citation percentage is a separate quantity. Inputs average 18.28 pages and 2.16 evidence regions per QA. Results use the same backbone and training data.
Region-F1 is conditioned on exact page-set matching. Table 1 general-purpose VLMs reach at most 7.08 Region-F1 and 3.21 Typed-F1. Appendix D illustrates evidence errors missed by answer-only scores.
4. Ground-then-classify baseline
Our modular baseline predicts answers/regions using Default SFT, then classifies relations with a ReViE-trained two-layer MLP on frozen Qwen3-VL-8B features. Inference uses predicted boxes and cannot revise them.
VisProve exceeds Two-stage by 3.77 Region-F1, 3.67 Relation-F1, and 1.72 Typed-F1 points. Table 3 compares existing methods under a shared output format.
5. Human verification
Two external annotators independently relabeled 180 balanced regions, blinded to original labels. Agreement was 150/180 (83.33%), Cohen's κ = 0.750, with positive agreement of 83.48% / 78.26% / 87.69% for Quotation / Compression / Inference. After third-annotator review of 42 disputes, 163/180 (90.56%) original labels had two independent supporting votes in this sample.
GPT-5.6 Sol assisted evidence refinement and relation labeling during construction. End-to-end QA requires predicting answers and evidence from full documents. Our comparison uses the same Qwen3-VL-8B backbone and SFT training data.
6. SER terminology
SER organizes each page's visual features into spatially encoded 16 × 16 grids. We will replace “memory” with “Visual Feature Bank” in Section 4.2 and Appendix B.2 to avoid ambiguity.
7. LLM-as-Judge agreement
Fig. 5's r = 0.9898 uses 11 model-level means. For item-level agreement, we compared 220 original model–question ratings (11 models × 20 questions, 197 unique answers) with median human scores on a 0–5 scale, without rescoring.
CIs use 3,000 question-clustered bootstrap resamples. The Judge scores answer text only and is excluded from Overall.
8. Dataset scale and supervision
ReViE spans eight domains with document-disjoint splits, 3,941 QAs (3,115 training), and 8,523 typed evidence regions (6,738 training).
Held-out gains demonstrate the value of structured supervision. We will clarify supervision density in the revision.
We hope the modular baseline, human verification, and item-level agreement analysis strengthen our findings and address your concerns. We respectfully ask you to reconsider your assessment.