acceptodds
Under review as a conference paper at ICLR 2027

Verifiable Paper-to-Video Generation

Abstract

Modern paper-to-video systems can now produce polished and compelling presentations from scientific papers. Yet visual quality does not guarantee faithfulness to the source. Existing evaluations focus largely on presentation quality and informativeness, with much less attention to whether the claims in a generated video are actually supported by the paper. We introduce PRESENTFAITH, a benchmark that compares claims recovered from the generated video with a fine-grained Evidence Index of the source paper. Our evaluation shows that current systems can perform well on standard presentation metrics while still producing substantial unsupported and contradictory content. These findings motivate a change in how the presentation itself is generated. We propose EVI-IR, an evidence-linked intermediate representation that stores each generated claim in a structured form together with its claim type, evidence pointer, and relevant quantities. Keeping this information explicit makes each claim traceable to the paper and its supporting evidence directly verifiable. EVI-IR then checks these links whenever possible and repairs unsupported or inconsistent claims before rendering. Experiments on PRESENTFAITH demonstrate that EVI-IR reduces the unfaithful claim rate from 17.6% to 12.5% relative to the strongest baseline, while increasing supported claims per minute from 11.7 to 13.6 and preserving existing presentation-quality metrics.

Primary area:
datasets and benchmarks
Keywords:
verifiable generation, paper to video, presentation generation, faithful evaluation, intermediate representation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Z8kJOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the issue of content faithfulness when automatically generating presentation videos from research papers. The authors note that while existing systems can produce fluent and complete videos, they may present claims unsupported by the source paper or misstate numerical values ​​and comparative relationships. To address this, the paper introduces the PRESENTFAITH benchmark, which extracts atomic claims from video narrations and slides, then verifies them against evidence indexed within the paper—down to the level of individual table cells. Furthermore, the authors propose the EVI-IR generation framework, which tracks the evidence source for every generated claim and identifies and corrects inconsistencies prior to video rendering. Experiments conducted on 40 research papers demonstrate that, compared to the strongest baseline, EVI-IR reduces the proportion of unfaithful claims from 17.6% to 12.5% ​​and numerical discrepancies from 9.7% to 2.6%. Simultaneously, it increases the number of paper-supported claims per minute while maintaining the overall quality of the presentation.

Critical strengths
  1. Build fine-grained fidelity reviews for final videos. It checks the assertions in the narration and slides one by one, and locates the numbers to specific table cells in the paper, thereby discovering the problem of "the numbers exist, but the attribution or comparison relationship is wrong."
  2. Move evidence verification earlier into the video generation process. Each assertion is associated with the original text evidence when it is generated, and is rendered after being checked and partially corrected. Experiments show that this reduces claims of disloyalty while maintaining video presentation quality.
Critical weaknesses
  1. The "verifiable" scope of the method is limited. EVI-IR’s numerical checks primarily address relationships between, at most, two evidence units; broader conclusions drawn across multiple experimental setups, information embedded within figures, and descriptive assertions still largely rely on the judgment of language models. The paper reports that 12.5% ​​of the assertions regarding the videos remain unfaithful, yet it does not quantify the specific proportions of the various remaining types of errors.
  2. Experiments are not yet sufficient to support broad applicability. The 40 test papers were screened and all were numerically dense English papers; the main experiments all used LaTeX source files. The paper acknowledges the degradation in quality of evidence indexing resulting from PDF parsing, but does not test this input in the main experiment. Therefore, there is currently no evidence whether papers with fewer tables, relying on complex graphics to express results, or only PDFs will achieve similar benefits.
Questions for authors
  1. On the same batch of manually annotated video judgments, compared with existing visual language model evaluation methods, how many more errors of "numbers correspond to wrong objects" or "comparison directions indicate the opposite" were identified by PRESENTFAITH, and how many more false positives were added?
  2. How will EVI-IR's proportion of unfaithful assertions, evidence pointer accuracy, and abstention rate change if the input is only PDF, or if the paper's conclusions are mainly supported by complex graphs and multiple experimental results?
2 comments
AuthorsOpenReview ↗

Thanks Reviewer Z8kJ for the helpful suggestions

We greatly appreciate Reviewer Z8kJ's helpful comments, which has helped us to further refine this work.

>>> W1: Scope of verification and breakdown of error types

>>> A1:

(a) Scope of numerical verification.

Currently, our numerical checks do involve at most two evidence units, and some broader conclusions still rely on the judgment of language models. We deliberately keep the checks simple because this is the first work to verify paper-to-video generation content at the claim level. We chose a reliable and simple form - pairwise comparison. It has already proven to be superior to previous work, so we did not delve further into this aspect in this paper.

However, we fully agree with your suggestion. As stated in the limitation section, we will support more numerical types of checks in subsequent work and improve the numerical verification mechanism, , which is very meaningful.

(b) Breakdown of error types.

We thank the reviewer’s finding .As requested, we have reported the specific proportions of EVI-IR's unfaithful claims rate (UCR 12.5%) as Table 11 (Appendix C.8) and summarize it here:

Setup 

We collected all 1,442 unfaithful claims from EVI-IR's delivered videos. Each claim was traced back to its IR atom (pointer, type, V1–V4 outcome, repair history). Two non-author annotators then assigned it a single error type from a taxonomy fixed before labeling; a third resolved disagreements (κ = 0.74).

Results

GroupError typenShare (%)UCR pts
Beyond symbolic layerRelations over ≥3 evidence units40928.03.5
Descriptive over-statement33123.22.9
Figure-only information29120.02.5
Within symbolic layerNear-miss cell664.80.6
Reversed comparison372.40.3
Measurement & renderingVerifier false positive15210.41.3
ASR/OCR/segmentation1138.01.0
Rendering alteration221.60.2
Other—211.60.2
Total1,44210012.5

It can be seen from the table that the residual is dominated by the scope of verification (71%, 8.9 pts). And errors inside the symbolic layer total only 0.9 pts. In addition, about 20% consists of verifier false positives, perception errors, and rendering alteration.

>>> W2 Broad applicability

>>> A2:

Thanks for your suggestions, it’s very important for us. We will make the following explanations:

(a) For the dataset diversity.

We selected 40 papers to avoid including unrepresentative ones and ensure that the dataset is not overly influenced by certain special articles in the evaluation results. Although the screening process was conducted, the complexity and focus of these 40 papers vary, and there are also some papers with fewer tables and relying on complex graphical representations to express their results. Table 1 reports these differences, which to some extent reflects the robustness of EVI-IR. However, we fully agree with your suggestion to explore more complex situations, as it is meaningful. In the next study, we will collect more articles in the dataset that rely on complex charts as conclusions. Additionally, we also plan to expand the data scale and domain diversity in the same way.

(b) For PDF input.

Our system supports PDF input. Our focus is on the link between the paper itself and the generated claims, while PDF introduces an extra OCR step and thus extra loss, so the main experiments only reported the better LaTeX-based results. As requested, we report the evaluation results with PDF as input below (for clarity, we also copy the LaTeX-based results of EVI-IR):

System(Input)UCR ↓NCR ↓EPM ↓SPM ↑Sim. ↑Arena ↑Quiz ↑IP ↑
EVI-IR (LaTeX)12.52.61.913.60.600.180.960.55
EVI-IR (PDF)14.13.92.213.20.600.170.950.53

The results show that the evaluation result of PDF as the input has a slight decrease compared to LaTeX, but the overall performance is still good. So the generation quality of EVI-IR is not significantly affected by the input type.

>>> Q1: Detailed of the PresentFaith Results

>>> A3:

Thanks for your questions and it is valuable.

Supplementary experiment

Our verifier already agrees with the human majority on 93.4% of claims. However, as requested, we conducted additional experiments to compare its results with those of visual language model: We replaced the step in PRESENTFAITH where it determines whether a paper supports a claim in a video with a visual language model judge (Gemini-2.5-Flash). Then, on the same manually annotated sentences, we compared the two and the results are as follows:

Human-labeled categoryclaimsVLM judge detectsPRESENTFAITH detects
Number bound to the wrong object (row/method/setting)7127 (38.0%)63 (88.7%)
Reversed comparison direction4619 (41.3%)41 (89.1%)
Other unfaithful claims135104 (77.0%)113 (83.7%)
False positives on clean claims (n = 988)—13 (1.3%)16 (1.6%)
Agreement with human majority (all 1240)—85.1%93.4%

Summary

On the same claims, PRESENTFAITH finds 58 more wrong-object and reversed-comparison errors (+36 and +22) than the VLM judge, at the cost of 3 more false positives (1.3% → 1.6% FPR). The VLM judge misses most of these relational errors because every value in them does appear in the paper, so a judge that checks for the value's presence accepts the claim. This matches the injected-error study in Appendix B.3 (38% / 31% detection). PRESENTFAITH instead looks up the cells the claim names and recomputes the relation. We have added this comparison to Table 14 (Appendix C.11).

>>> Q2: Evaluation of EVI-IR for different inputs

>>> A4:

(a) PDF input

As we mentioned in A2-(b), we have reported the evaluation results of pdf as the input.

(b) Complex papers

Experimental Setup

Thank you very much for your points out the shortcomings. Full annotation is costly, so we selected 3 additional papers outside PRESENTFAITH. In each, more than half of the key claims in the abstract and conclusion are supported only by figures or by three or more experimental results. We ran EVI-IR and PaperTalker on them. In addition, we also ran PaperTalker on these papers, in order to facilitate the comparison of the variation range. We have added the details in Table 15 (Appendix C.12).

SettingSystemUCR↓NCR↓Ptr. acc.↑Abstain (%)
PRESENTFAITH (40 papers)PaperTalker17.69.7——
EVI-IR12.52.684.612.0
Complex (3 papers)PaperTalker26.911.8——
EVI-IR19.83.481.721.5

Summary

  • Both systems degrade on these papers, and EVI-IR still keeps a clear margin. UCR rises 12.5 → 19.8 (per-paper range 17.2–22.6), and the gap to PaperTalker is 7.1 points. NCR barely changes, because table-backed numeric claims are still checked symbolically.
  • Most of the increase comes from where the symbolic layer does not reach. Of the 7.3-point UCR increase, figure-only information accounts for 4.1 points and relations over three or more units for 2.2. These are the same two categories identified in Appendix C.8.
  • The pointers that are emitted stay accurate, while abstention nearly doubles. Claims about figure internals have no index unit to point to, so the system abstains instead of emitting a wrong pointer. Abstained atoms are still rendered without a binding check, however, and they are the main source of the higher UCR.

These results point to two extensions in the future: evidence units for figure internals and n-ary relation checks. We have added both to the Limitations section as future work.

Reviewer_WNt7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on paper-to-video generation, specifically whether the claims in generated presentation videos are actually supported by the source paper. To measure this, the authors introduce PresentFaith, a benchmark that extracts claims from a generated video's narration and slides and checks them against an index of the paper's content. They also propose EVI-IR, a generation method where each claim carries a pointer to its supporting evidence, and these links are checked and repaired before the video is rendered. On 40 papers, EVI-IR lowers the unfaithful claim rate from 17.6% to 12.5% and numeric contradictions from 9.7% to 2.6% compared with PaperTalker, while keeping presentation quality comparable.

Critical strengths
  • This paper addresses a timely problem; paper-to-video generation is becoming more widely used, but current metrics primarily look at presentation quality and don’t consider the accuracy of the information being presented. Breaking videos into atomic claims and indexing evidence down to individual table cells is well-motivated by the existing fact-checking and grounding literature.
  • The benchmark is well validated, with strong agreement between the verifier and human labels and a test with deliberately injected errors, where the verifier matches or exceeds human annotators.
Critical weaknesses
  • The paper argues that linking claims to evidence during generation works better than verifying content after it's written. However, none of the baselines verify faithfulness at all, so Table 1 can't separate the benefit of having a faithfulness check from the benefit of building it into generation. In addition, OmniPresent (Ma et al., 2026) is described as the closest prior system, since it already plans from an evidence stream and verifies plans before rendering, but it isn't included as a baseline. The claimed advantage of cell-level pointers and symbolic checks over its VLM-judged verification is therefore not shown experimentally.
  • The dataset is small, with only 40 paper-video pairs from a narrow domain (English ML, CV, and NLP papers), so it's unclear how well these results generalize.
Questions for authors
  • The related work should discuss citation-aware RAG evaluation beyond ALCE (Gao et al., 2023b). ARGUE (Mayfield et al., 2024) tests whether each sentence is supported by the documents it cites, similar to the paper's pointer-level verdict, and RAGAS (Es et al., 2024) splits answers into statements and scores each against the retrieved context, much like the existence-level UCR. MIRAGE (Martin et al., 2025) adapts ALCE, ARGUE, and RAGAS to multimodal sources and finds that subclaim-level evaluation agrees with human judgments better than the sentence-level checks in ARGUE and ALCE, which supports this paper's use of atomic claims.
4 comments
AuthorsOpenReview ↗

Thanks Reviewer WNt7 for the constructive feedback

We thank Reviewer WNt7 for the thorough reading and the valuable suggestions.These suggestions are extremely valuable to us, and we have addresses each point below.

>>> W1: Arguments for faithfulness verification & Baseline

>>> A1:

(a) Arguments for faithfulness verification

Thank you for this careful comment. We agree that, because none of the existing paper-to-video baselines includes a faithfulness check, Table 1 alone cannot show where EVI-IR's gain comes from. We address them in turn: (i) with our existing ablation, and (ii) with a new controlled experiment.

(i) Evidence-linked generation helps even without any check. In the ablation (Table 2), removing the verifier leaves EVI-IR at 15.0 UCR, already below PaperTalker (17.6). Further removing the evidence links raises UCR to 16.2, and removing the IR as well brings it back to PaperTalker's level (17.9). Generating with explicit evidence links therefore improves faithfulness before any verification is applied.

(ii) Placement of the faithfulness check matters. If the source pointer is retained during the generation process, i.e. an intermediate representation with a point-to-point correspondence between each claim and its evidence, a one-to-one judgment can be made during the verification stage as to whether it supports. But if verification is done after generation, these pointing information is lost, and one can only search the entire article to determine if the claim is supported. Therefore, theoretically, the pre-check is more precise.

Since the other baselines have’t any intermediate representation and evidence pointers and did not have a specific verification mechanism for faithfulness, it was difficult to confirm the above conclusions through direct comparison. Therefore, we add a LLM-based post-hoc check for comparison, which is a simple and direct approach that can also avoid additional interference. To be specific, an LLM (same GPT-4.1 backbone) is given the full paper and the generated script, flags unsupported or contradicted claims, and triggers the same localized repair loop. We apply it to two generators: PaperTalker, and EVI-IR's own generator with the emitted evidence pointers withheld from the checker.

GeneratorVerificationUCR↓NCR↓SPM↑
PaperTalkernone17.69.711.7
PaperTalkerpost-hoc LLM15.17.611.9
EVI-IR generationnone15.08.413.3
EVI-IR generationpost-hoc LLM13.66.313.4
EVI-IR generationpointer-based (ours)12.52.613.6

Findings:

  1. Verification is useful. All checks lowers UCR for both generators (17.6 → 15.1 and 15.0 → 13.6).
  2. With the generator fixed, building the check into generation helps further. Pointer-based checking outperforms post-hoc checking on the same generator (UCR 12.5 vs. 13.6; NCR 2.6 vs. 6.3).
  3. With the checker fixed, evidence-linked generation helps on its own. Under the same post-hoc check, EVI-IR generation still beats PaperTalker (13.6 vs. 15.1 UCR).

We have added the detailed experiment in Table 12 (Appendix C.9).

(b) Baseline

Reason

Thank you for raising this point. We totally agree that OmniPresent should be served as a baseline. Unfortunately, we checked the relevant materials of OmniPresent and found that they have not released the code and any demo video either. Only the slide pages of five papers were released, so we can‘t fairly include it in the baselines for comparison.

Experiment

Nevertheless, we still supplemented the relevant experiments as much as possible. Specifically, we extracted the bullets from each slide page. Then, we constructed the Evidence Index for these five papers, following the same process as for other papers in our dataset. We evaluated the faithfulness metrics by PRESENTFAITH (since slides lack information such as timelines and narration compared to videos, we can only evaluate the UCR and NCR for the bullets in the slides). Additionally, we ran these five cases as inputs in EVI-IR and PaperTalker. To be fair, we only kept the slides and Conducted the same evaluation as OmniPresent to evaluate the metrics. The experimental results are as follows:

SystemUCR ↓NCR ↓
OmniPresent15.36.1
PaperTalker18.49.2
EVI-IR10.12.4

The results show that EVI-IR achieves a clear advantage over OmniPresent (UCR 10.1 vs. 15.3, NCR 2.4 vs. 6.1), which supports the claimed benefit of cell-level pointers and symbolic checks over VLM-judged verification; the gap is largest on NCR, where numeric claims are decided by exact cell-level arithmetic.

If OmniPresent releases the code or presentation videos in the future, we would be happy to add it to baselines, which would make our experiments more complete.


>>> W2: Dataset size and domain

>>> A2:

We are pleased for your suggestion. We totally agree that 40 papers is small by general ML standards.

But within paper-to-video generation task, this scale is comparable to other baselines: PresentAgent(30), Paper2Video(101), MPresenter(40).

And the main cost in our setting is building a cell-level Evidence Index and human label for each paper, so we set the number of the dataset to 40. In terms of the field, most of the work selected papers from the fields related to computer science as the dataset. Therefore, we also chose three of the most important subfields from among them.

Expansion Plan:

However, we believe this suggestion is extremely meaningful. More datasets and a broader range of fields will help enhance the generalizability of our work.

Therefore, we will plan to expand the dataset in our next work. Our concrete expansion plan:

In terms of quantity, we will collect papers with human-produced videos that have been newly released, and our initial goal is to expand by more than double.

In terms of field, we will expand the dataset to other areas of computer science, such as: software engineering, robotics, computer networks, etc., to ensure that the dataset has sufficient diversity.


>>> Q1: Related work beyond ALCE: ARGUE, RAGAS, MiRAGE

>>> A3:

We thank the reviewer for these references. We now cite and discuss all three in Section 2; several of their findings are consistent with our design choices.

  • RAGAS splits answers into statements and checks each against the retrieved context. Our existence-level verdict (any unit supports the claim, which underlies UCR) is its analogue.
  • ARGUE verifies sentence-level support against cited documents; our verdict is the cell-level analogue, and both rely on human validation to avoid circular evaluation.
  • MiRAGE finds that subclaim-level evaluation matches human judgments better than sentence-level, which supports our use of atomic claims. It also pairs precision with coverage, as we pair UCR with SPM.

PRESENTFAITH differs from all three in three ways. First, its claims must be recovered from a video through ASR and OCR. Second, it measures informativeness against the source paper. Third, it verifies numeric and comparative claims arithmetically against cell-level evidence.

>>> Reference

[1] Ma, Q., Xiao, J., Wang, S., Tian, Z., Feng, W., Wang, S., ... & Zhang, Z. (2026). OmniPresent: Generating Coherent Presentation Suites from Scientific Papers. arXiv preprint arXiv:2607.02590.

[2] Gao, T., Yen, H., Yu, J., & Chen, D. (2023, December). Enabling large language models to generate text with citations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 6465-6488).

[3] Mayfield, J., Yang, E., Lawrie, D., MacAvaney, S., McNamee, P., Oard, D. W., ... & Hibbler, N. (2024, July). On the evaluation of machine-generated reports. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 1904-1915).

[4] Es, S., James, J., Anke, L. E., & Schockaert, S. (2024, March). Ragas: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th conference of the european chapter of the association for computational linguistics: system demonstrations (pp. 150-158).

[5] Martin, A., Walden, W., Kriz, R., Zhang, D., Sanders, K., Yang, E., ... & Van Durme, B. (2026, October). Seeing through the mirage: Evaluating multimodal retrieval augmented generation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (pp. 40231-40254).

Thank you for the additional results, which alleviates one of my primary concerns. I have updated my review to a weak accept.

AuthorsOpenReview ↗

Thanks for Reviewer WNt7

Thank you so much for your kind words and for taking the time to review our responses and experiments. We are extremely grateful for the positive feedback! Your feedback and support mean a lot to us!

Reviewer_LJfvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies faithfulness in automatically generated scientific presentation videos. The authors introduce PRESENTFAITH, a benchmark that evaluates atomic claims extracted from generated videos against a fine-grained Evidence Index constructed from the source paper. The benchmark distinguishes supported, contradicted, and unsupported claims and introduces metrics that measure both error rates and supported information density.

The paper further proposes EVI-IR, an evidence-linked intermediate representation that associates generated claims with evidence pointers, claim types, and numerical quantities before rendering. A layered verification mechanism checks evidence resolution, type compatibility, numeric consistency, and semantic entailment, followed by localized repair of incorrect claims.

Critical strengths
  1. The paper addresses a meaningful limitation of existing paper-to-video systems: presentations can appear visually convincing while communicating unsupported scientific claims. Evaluating faithfulness independently of presentation quality is valuable, especially for scientific communication.

  2. The Evidence Index supports verification at the level of individual table cells, including their row and column context. This is useful for identifying numerical misattributions and reversed comparisons that may be missed by document-level semantic similarity or entailment-based evaluation.

  3. The reported improvements in the metrics support the claims and contribution of the paper. The inclusion of human audits, ablation studies is helpful to decompose where the gains coming from. However, there needs to be more clarification and rederign evaluation.

Critical weaknesses
  1. Insufficient comparative baselines, for example they say OmniPresent is the closest prior work however the same prior work is absent in the eval tables.
  2. Missed related works, especially the section on the citation generation and attribution is outdated (it list all the methods from 2023), there are several methods even iterative verifications that are proposed more recently - some on top of my head - https://aclanthology.org/2026.magmar-main.14/, https://aclanthology.org/2026.magmar-main.15.pdf. similarly the recent works for evaluation are also not discussed: Seeing Through the MiRAGE: Evaluating Multimodal Retrieval Augmented Generation introduces InfoF1 and CiteF1 for evaluating factual content and citation support - https://arxiv.org/abs/2510.24870 EMNLP 2026.
  3. All evaluated indices are constructed from LaTeX sources? if so practical paper-to-video systems frequently receive PDFs. Appendix A.1 identifies PDF-derived indices as degraded, but their effect on downstream performance is not evaluated.
  4. Writing is weak, there are many section where it seems use of LLMs for writing has distorted the main contributions for example : The presentation should more explicitly distinguish the benchmark's post-hoc retrieval process from the proposed framework's evidence-linked generation process. This distinction is central to understanding the contribution. Also, fig2 is also poorly organized waht is the plot supposed to present SPM vs EPM?
  5. Limited evaluation of renderer robustness. The proposed framework fundamentally relies on preserving verified claims during rendering. No experiments investigate whether the observed faithfulness improvements persist with alternative renderers, particularly LLM-based or generative rendering pipelines that might modify narration, slide text, or visual content. Additionally, the specific renderer implementation is insufficiently documented, making reproducibility difficult. Evaluating renderer variations or a stronger post-render verification setting would help establish the robustness of the proposed approach.
Questions for authors

Please check weakness.

4 comments
AuthorsOpenReview ↗

Thank Reviewer LJfv for the valuable comments

We are sincerely grateful to Reviewer LJfv for the detailed and constructive comments, which have helped us substantially improve the manuscript. We address each point in turn below.

>>> W1: Comparison with OmniPresent

>>> A1:

We thank the reviewer for raising this point. We agree that a comparison with OmniPresent would be informative. However, a direct comparison with OmniPresent is not feasible, as its code and full generated videos are unavailable. The authors provide only slides for five papers, none of which are included in our benchmark. We therefore exclude OmniPresent from the main comparison table to ensure a consistent evaluation protocol.

Experimental Setup

As suggested, we built the Evidence Index U(P) for these five papers. We then manually extracted the bullet points from each slide and scored them with the same verdict pipeline of PRESENTFAITH . We also applied EVI-IR and PaperTalker to the same five papers and scored its slides in the same way. Because slides lack the temporal dimension of videos, we report only UCR and NCR (lower is better):

SystemUCR ↓NCR ↓
OmniPresent15.36.1
PaperTalker18.49.2
EVI-IR10.12.4

Results

The results are consistent with our main findings: EVI-IR outperforms OmniPresent on both UCR and NCR, although the small sample and the restricted metric set limit how far this comparison generalizes. We report this comparison separately in Table 8 (Appendix C.5) and will add OmniPresent to the main table once its code or full generated videos are available.


>>> W2: Related work

>>> A2:

We thank the reviewer for bringing these works to our attention. They are closely related and we have already incorporated this discussion and cited all three in the revised manuscript (Section 2).

  • TRACE grounds before reasoning: For each video, it builds a timeline from OCR and detection results. A text-only LLM then locates the supporting evidence and writes claims that cite other videos.
  • CRAFT refines atomic claims through a hybrid critic loop and shows that specialized checkers outperform a general-purpose judge.
  • MiRAGE evaluates multimodal RAG with InfoF1 and CiteF1 at the claim level.

These works share our core view that individual claims should be grounded and verified. However, our work differs in two key aspects: the task setting and the evaluation protocol.

  • Task. All three operate on existing videos, documents, or retrieved multimodal contexts and produce or assess textual claims. We generate video from a paper, so claims must first be recovered from the generated video through ASR and OCR.
  • Evaluation. Their verdicts are produced by learned models, and MiRAGE's recall requires reference answers. PRESENTFAITH verifies numeric claims arithmetically against the source paper.

>>> W3: PDF input evaluation

>>> A3:

Yes, all evaluated indices are from LaTeX. But EVI-IR also supports PDF inputs.

Reason

In the main paper, we use LaTeX because our evaluation verifies each claim against exact table cells. This requires a one-to-one mapping between claims and supporting evidence. LaTeX provides complete information and preserves the original table structure. In contrast, PDF parsing can introduce errors, such as OCR mistakes and lost table structure. Using LaTeX allows us to evaluate our method without these errors.

Results

We now also report results with PDF inputs in the following table:

System(Input)UCR ↓NCR ↓EPM ↓SPM ↑Sim. ↑Arena ↑Quiz ↑IP ↑
EVI-IR (LaTeX)12.52.61.913.60.600.180.960.55
EVI-IR (PDF)14.13.92.213.20.600.170.950.53

With PDF input, EVI-IR's faithfulness degrades moderately (UCR 12.5→14.1, NCR 2.6→3.9), and NCR shows the largest relative increase, which is expected because cell-level pointers depend directly on table parsing. The presentation-quality metrics (Sim., Arena, Quiz, IP) change by at most 0.02 relative to the LaTeX setting. This indicates that the gain of EVI-IR is not an artifact of using LaTeX inputs. We have added these results to Table 9 (Appendix C.6).

>>> W4: Writing and Figure

>>> A4:

Thank you for the suggestions on the writing and Figure 2.

(a)Writing

We are sorry we didn’t make it clearer. We have revised several passages in the paper and made the key points clearer. Below we address the specific points:

  • benchmark's post-hoc retrieval process: PRESENTFAITH judges a generated video post hoc. It recovers claims via ASR and OCR, retrieves candidate evidence units, and checks whether any unit supports each claim.
  • proposed framework's evidence-linked generation process: EVI-IR instead writes each claim together with its pointer at generation time, checks whether the pointed-at unit supports it before rendering, and repairs the claim locally if the check fails.

The PRESENTFAITH judge never sees EVI-IR's pointers, so all systems are evaluated under the same protocol.

(b) Fig. 2

We are truly sorry for the confusion. Figure 2 mainly shows how PRESENTFAITH computes the faithfulness metrics from a presentation video. We have revised Figure 2 to make it easier to understand. Specifically, we have provided clearer descriptions of the four metrics in the right panel and added a brief explanation of the (SPM, EPM) plane in the caption.


>>> W5: Renderer robustness and specific implementation

>>> A5:

Thanks for raising this point. Our original submission reported results only with the default renderer. We now add a controlled robustness study and the full implementation details:

Clarification on "LLM-based / generative rendering"

In EVI-IR, slides are deterministically compiled from the verified IR / written by the LLM as LaTeX or HTML after verification and then turned into pages by an external compiler (pdflatex/xelatex or a browser). Speech is produced by a TTS engine. Importantly, PresentFaith reads claims back from the delivered video (OCR on frames, ASR on audio), so any error introduced after verification is already counted in UCR/NCR.

Experiment

We re-rendered the same verified IRs for all 40 papers, changing one back-end at a time, and re-scored the resulting videos. Claim loss is the fraction of claims verified in the IR that are missing from, or altered in, the read-back of the final video.

Slide back-endSpeech back-endUCRNCRClaims lost/altered in rendering
pdflatex, TeX Live 2024F5-TTS12.52.60.4%
xelatex, TeX Live 2026F5-TTS12.62.60.4%
HTML / reveal.jsF5-TTS12.92.80.9%
pdflatex, TeX Live 2024Kokoro-82M13.02.91.1%

It can be seen from the table that at least 98.9% of verified claims survive in every setting, and UCR/NCR change by at most 0.5/0.3 points. The two LaTeX engines give identical claim loss. The extra loss with reveal.js comes from layout effects, e.g., text overflow in dense tables, and the extra loss with the Kokoro-82M comes from verbalization.

We have added the full rendering configuration in Table 4 (Appendix C.2) and this study in Table 10 (Appendix C.7).

>>> Reference

[1] Ma, Q., Xiao, J., Wang, S., Tian, Z., Feng, W., Wang, S., ... & Zhang, Z. (2026). OmniPresent: Generating Coherent Presentation Suites from Scientific Papers. arXiv preprint arXiv:2607.02590.

[2] Yan, P., Gorugantu, A. V., Bhosale, M., Wasi, A., Trivedi, V., & Doermann, D. (2026, July). TRACE: Evidence Grounding-Guided Multi-Video Event Understanding and Claim Generation. In Proceedings of the 2nd Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2026) (pp. 120-129).

[3] Bhosale, M., Wasi, A., Trivedi, V., Yan, P., Gorugantu, A. V., & Doermann, D. (2026, July). Craft: Critic-refined adaptive key-frame targeting for multimodal video question answering. In Proceedings of the 2nd Workshop on Multimodal Augmented Generation via Multimodal Retrieval (MAGMaR 2026) (pp. 130-143).

[4] Martin, A., Walden, W., Kriz, R., Zhang, D., Sanders, K., Yang, E., ... & Van Durme, B. (2026, October). Seeing through the mirage: Evaluating multimodal retrieval augmented generation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (pp. 40231-40254).

[5] Zhu, Z., Lin, K. Q., & Shou, M. Z. (2025). Paper2video: Automatic video generation from scientific papers. arXiv preprint arXiv:2510.05096.

AuthorsOpenReview ↗

Response to Reviewer LJfv

Dear Reviewer LJfv,

Thank you for your time on this paper. As the rebuttal deadline approaches, we would greatly appreciate your support and positive feedback. Please let us know if there is anything further we can provide from our side.

Best regards,

Authors of the paper 10491

AuthorsOpenReview ↗

Response to Reviewer LJfv

Dear Reviewer LJfv,

Apologies for the additional reminder. With the rebuttal period closing in a few hours, we would greatly appreciate it if you could take a brief look at our responses. If any concerns remain, we would be happy to clarify them in the time available.

Thank you again for your time and consideration.

Best regards,

Authors of the paper 10491

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank all and Summary of Discussion

Pre rebuttal

We sincerely thank all reviewers for their thoughtful and constructive feedback. We are encouraged that the reviewers agree on the following core contributions:

  • Important and timely problem: Reviewer LJfv, Reviewer WNt7, Reviewer Z8kJ. Particularly, Reviewer LJfv noted that evaluating faithfulness is "valuable, especially for scientific communication."
  • Fine-grained, evidence-grounded evaluation design: Reviewer LJfv, Reviewer WNt7, Reviewer Z8kJ
  • Novel and well-motivated method design: Reviewer LJfv, Reviewer Z8kJ
  • Convincing experimental results: Reviewer WNt7, Reviewer Z8kJ
  • Reliable and well-validated benchmark: Reviewer LJfv, Reviewer WNt7

Taken together, the reviewers see this work as offering both a means to measure faithfulness in generated scientific presentations and a practical approach to improving it. We are extremely grateful and delighted to see that all reviewers rate soundness as having either no substantive concerns or only minor, fixable ones. We respond to each reviewer individually and have revised the paper accordingly.

After rebuttal

Following our additional experiments, Reviewer WNt7 updated their recommendation from weak reject(2) to weak accept(3) , stating that the new results “alleviates one of my primary concerns.”. Overall, our rebuttal strengthens the evidence primarily in following areas:

  • Adding other verifier to compare (Reviewer WNt7). With the generator fixed, pointer-based verification reduces unfaithful claims from 13.6% to 12.5% and numeric contradictions from 6.3% to 2.6% compared with a post-hoc LLM checker.
  • Fine-grained evaluation validated (Reviewer Z8kJ). On the same human-annotated claims, PRESENTFAITH detects 58 more wrong-object or reversed-comparison errors than a VLM judge, with only 3 additional false positives.
  • Broader comparisons and robustness evidence (Reviewer LJfv, Reviewer WNt7, Reviewer Z8kJ). We add a five-paper, slides-only OmniPresent comparison and PDF-input evaluation, alongside alternative rendering back-ends.

In addition, we have also supplemented other details to address the reviewers' concerns and improve our paper.

Finally, thanks all Reviewers and Area Chairs again for the help and suggestions, which extremely means a lot to our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.